Tag: language modeling

Against Superheroes: Z Collective Commentary, Section 1

Mark Davis1 Comment

Author’s Note: This is the first chapter of my novel, Against Superheroes, in its original form. It was conceived as an analysis of a tract found in space by aliens. The goal was to write from an exotic analytical perspective that misinterprets and overanalyzes the contained story, but that also contains a story in itself about the possibility that the inner story is related to the aliens’ culture. Footnotes are in their original alien translations, per the directives of my studious overlords.

Section 1

Z3 begins with a fragment from Sinister’s earliest recollections of the initial transformation:

The fear began with the realization that my right arm was becoming unusually heavy. The weight of the bracelet had not changed dramatically, but it seemed that my arm was thickening and I feared I would lower my arm and the combined artifact would slip off, risking possible damage on the tile floor, and so I reflexively swung my left arm to stabilize my wrist. The blank, formless face of the figure was less tarnished than the rest and the dim bathroom light dancing across the visage gave it a strangely animated swirling quality. Soon the weight in my arm moved through my shoulders and into my neck. I staggered and dropped to my knees.

All Z collectives know this passage, but we disagree with Z2’s reading in Peregrinations of Mythic Specialness1 that the inclusion of the specific details concerning the type of light amongst the picturesque imagery in the passage is a deliberate effort on the part of later redactors to try to concretize a mythic passage. It is equally possible to simply conclude that the author was not concerned with the overall flow of the writing but instead intended to convey facts while capturing aspects of his internal state.… Read the rest

One Shot, Few Shot, Radical Shot

Mark Davis1 Comment

Exunoplura is back up after a sad excursion through the challenges of hosting providers. To be blunt, they mostly suck. Between systems that just don’t work right (SSL certificate provisioning in this case) and bad to counterproductive support experiences, it’s enough to make one want to host it oneself. But hosting is mostly, as they say of war, long boring periods punctuated by moments of terror as things go frustratingly sideways. But we are back up again after two hosting provider side-trips!

Honestly, I’d like to see an AI agent effectively navigate through these technological challenges. Where even human performance is fleeting and imperfect, the notion that an AI could learn how to deal with the uncertain corners of the process strikes me as currently unthinkable. But there are some interesting recent developments worth noting and discussing in the journey towards what is named “general AI” or a framework that is as flexible as people can be, rather than narrowly tied to a specific task like visually inspecting welds or answering a few questions about weather, music, and so forth.

First, there is the work by the OpenAI folks on massive language models being tested against one-shot or few-shot learning problems. In each of these learning problems, the number of presentations of the training data cases is limited, rather than presenting huge numbers of exemplars and “fine tuning” the response of the model. What is a language model? Well, it varies across different approaches, but typically is a weighted context of words of varying length, with the weights reflecting the probabilities of those words in those contexts over a massive collection of text corpora. For the OpenAI model, GPT-3, the total number of parameters (words/contexts and their counts) is an astonishing 175 billion using 45 Tb of text to train the model.… Read the rest