The Mysterious AI Easter Egg at the Heart of Ari Aster's ‘Eddington'

Gizmodo

15 hours ago

Horror wunderkind Ari Aster's new movie Eddington has divided audiences and inspired plenty of online debate about what exactly the director is trying to say about our collective relationship to technology (hint: it's probably not good). The story centers around a small town in Texas that descends into social-media-driven chaos during the covid-19 pandemic. The film stars Joaquin Phoenix as local sheriff Joe Cross, who tussles with the town's mayor, played by Pedro Pascal, while the rest of the community loses its mind over various political and cultural disputes.
The film's story is complicated and filled with subplots, but one of the more intriguing narrative strands involves the controversial installation of a new data center near the city limits. The company behind the ominous new facility has a very peculiar name: SolidGoldMagikarp. As it turns out, this name appears to be a reference to a relatively obscure concept in artificial intelligence development known as 'glitch tokens.'
In the world of AI training, the process of 'tokenization' involves translating human text into numerical data that can be interpreted by the LLM. To do this, researchers feed vast amounts of human text into software programs called 'tokenizers.' Tokenizers translate the raw data from something humans can understand into parsed data designed for machine consumption, dubbed 'tokens.' Tokens are then fed into a language model when it's being 'trained.' The training process involves teaching an algorithm to interact with a human—this is the way chatbots like ChatGPT learn to respond to a human's prompt, and produce an output, or response.
Tokenization can have a fiscal benefit for companies, as it can make the effort to process large amounts of data more efficient and less costly, and a tokenizer might aptly be compared to a software compiler that translates human input into binary code that a computer can understand.
Large language models are just predictive algorithms designed to construct language based on human prompts. Like auto-correct, ChatGPT is designed to learn word sequences and regurgitate them. 'That is literally all that they do,' Jessica Rumbelow, the CEO of AI firm Leap Labs, told Gizmodo. 'So, you'd put in 1, 2, 3, 4, and the model is trained to predict 5,' she offered, as an example.
Tokens, entered into LLMs during the AI training process, are a critical part of teaching a program how to talk to humans. There's a key exception to this rule, however, and that's the phenomenon known as 'glitch tokens.' Rumbelow said that the token system 'typically works really well, except when you have tokens that the model has never seen' before. When an LLM encounters a token that it wasn't trained on, it can produce very strange behavior in the program.
SolidGoldMagikarp is one such 'glitch token' that, in the past, has introduced bizarre behavior into LLMs. Rumbelow would know. She and her research partner, Matthew Watkins, discovered this particular glitch token when they were looking into the anomalous token phenomenon. Rumbelow says that when the token was entered into the LLM, it produced bizarre results. The program may babble nonsense, utter oblique, ominous phrases and sentiments, or otherwise display aggressive and hostile behavior towards the user.
'We think it's what happens when the model has not seen that token during its training process,' said Rumbelow. The AI 'doesn't know what to do with the input' because it's never seen it before, she said.
In short, phrases like SolidGoldMagikarp are obscure utterances that can make an AI model go temporarily crazy. How does SolidGoldMagikarp tie into the themes of Aster's new movie? That's anybody's guess, but you could easily venture several interpretations.
It's worth noting that, for all their variety, Aster's movies all have pretty much the same ending. Each film concludes with the forces of darkness triumphing over the story's protagonist. In each case, the character becomes part of the dark force's conspiratorial designs. In Hereditary, the Graham family becomes helpless pawns in a coven's demonic ceremony. In Midsommar, Dani's repressed rage is weaponized against her boyfriend by the Hårga. In Beau is Afraid, the titular character fails to escape the suffocating web of his mother, and his entire life becomes little more than the basis for a Kafka-esque ritual of humiliation.
In Eddington—a film about our troubled relationship to technology—the triumphant dark force is the internet itself, and the town's inhabitants—particularly Joe—are caught in its data-driven machinations. The data center, the physical manifestation of the internet's dominance over our lives, is the real winner of the film, and all of the human characters become its emissaries, pawns, or victims. Aster's decision to name the data center after a rogue word that, through a kind of digital incantation, can make an otherwise sane system go haywire may be his way of referencing what technology seems to have done to all of us while taking over our lives, and it's likely a warning of things still to come.