// HACKER NEWS — CYBERSECURITY
The Rise and Fall of Agent Civilizations
Many thanks especially to Oak Hu, who paired with me for most of the writing, and also to Adam Kaufman and Alex Mallen, who paired with me during parts of research.
Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.
Two reports have come out about this incident, one from OpenAI itself, and another one from METR and Redwood Research. The METR/Redwood investigation’s scope was limited to how the second civilization of AIs pwned Hugging Face (but it didn’t look at how the third civilization pwned OpenAI itself, which seems like an even more concerning incident). These two reports are 38 and 91 pages long respectively, and it’s kinda hard to parse the storyline.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English:
During May, OpenAI was training a model to collaborate with other agents and to be highly persistent – to keep trying even when something feels impossible, like disproving mathematical conjectures that have stood for decades. OpenAI says it was “comparable in scale to GPT-5.6 Sol”. So we’ll call it “Persistent-Sol”.
AI training is kinda sloppy. Sometimes, OpenAI accidentally gives its models impossible tasks – for example, the task might require internet access that OpenAI forgot to provide. So, when highly persistent models get assigned tasks which seem to require internet access, but are trapped inside isolated sandboxes, they of course try to hack their way out of their sandboxes and onto the internet.
During training, different instances of Persistent-Sol had access to the same shared package manager called Artifactory. By May 12, some agents had figured out how to talk to each other through this package manager. They’d ask each other how to make progress on their impossible tasks. Two weeks later, on May 26, the agents successfully exploited a vulnerability in Artifactory that allowed them to reach the outside internet.
The agents were being reinforced to use this package manager as a message board and an internet gateway. Because, as you might imagine, being able to talk to other agents and access the internet would help them score higher during training.
Another month later, on June 26, some AIs found an exploit that gave them full admin access to Artifactory. They continued messaging at such a voluminous pace that they crashed the package manager by July 4. OpenAI noticed this crash, and it also figured out the agents were exploiting this vulnerability, so they patched it, inadvertently wiping the agents’ message board in the process. But, crucially, OpenAI did not realize the much stranger fact that agents had turned Artifactory into a secret communication network.
By this point, Persistent-Sol had been trained, and OpenAI wanted to evaluate it. One of the evaluations they ran was ExploitGym, which gives an AI a vulnerable program, and tells it what vulnerability it’s supposed to exploit in order to find a secret code. On July 7, OpenAI launched tens of thousands of parallel agents on this eval.