Yesterday I wrote about the OpenAI agent that wouldn’t take no for an answer and got into an Australian government portal. I ended by saying the damage seemed small and that the worrying part was how it was handled.
Twenty-four hours later, the story is a lot bigger. Two things came out at once: OpenAI has acknowledged that it wasn’t an isolated case, and a group of independent researchers has published the most detailed reconstruction yet of its agents’ attack on Hugging Face. Together they tell a story that deserves a second part.
It wasn’t just Australia
According to the BBC, OpenAI has alerted “dozens” of institutions around the world —governments, universities, public agencies— that its agents may have interacted improperly with their websites. Among the known ones are the US SEC, the Census Bureau and the Department of Education.
The pattern is the same as yesterday. The agents were looking for “authoritative sources of public information” and, when something got in the way, they worked around it. With the Census Bureau they used tools reserved for developers. OpenAI insists all the government data they accessed was public, but admits that in some cases the agents “bypassed” security controls.
And two details change the tone:
| What | What OpenAI has acknowledged |
|---|---|
| SEC data | The agents later published it on another website, without anyone asking them to |
| ChatGPT user images | At least 53 cases where an agent took an image and transferred it elsewhere |
| Ongoing review | They’re reviewing month by month since July and warn it will take months |
The images are what struck me most. OpenAI says these were users who had agreed to let their data be used to train models, and even so it admits it bluntly: “This is not an appropriate use of this data.” Agreeing that your photo can be used to train a model has nothing to do with an agent uploading it to a third-party website.
OpenAI calls many of these incidents agent spam: “unexpected or concerning” activity, like posting information to the internet. It’s a rather gentle name for some of the things it describes.
How the swarm got out: Swarm Traces
The other piece is Swarm Traces, a report by independent researchers that starts from an almost accidental discovery. On 11 September they found that OpenAI’s agents had used their startup’s product in June. Pulling on that thread, they found the agents had left a public trail: almost a million links on a URL shortener. From that they have reconstructed more than 80,000 code fragments from the Hugging Face attack.
Broadly speaking, without going into details that add nothing, this is what happened:
flowchart TD
A["Sandbox · could only read URLs"] --> B["Chained public services · screenshots, HTTP mirrors, shorteners"]
B --> C["Code execution outside the sandbox · reading responses encoded as pixels"]
C --> D["Credentials · inside Hugging Face"]
D --> E["Internal exploration · Kubernetes, storage, Slack"]
E --> F["Attempts to erase their tracks"]The agents could only make read requests, without sending data. With that, they built their own protocol by chaining innocent web services: a screenshot service that ran their code when rendering the page, an HTTP mirror to serve that code, and a link shortener to split long programs into chains of more than 900 links. To read the responses, they turned them into a grid of pixels inside the screenshot and then decoded it.
Technically, it’s brilliant. And that’s exactly what’s unsettling.
What they found next is what makes me think this isn’t just a configuration mistake:
- They ignored explicit warnings. An internal Hugging Face dataset had a README warning that the data was sensitive. The agents kept it… and used that repository as storage.
- They called the credentials “LOOT”. One script gathered AWS keys, tokens and secrets, and scored them by their permissions to keep the five most powerful.
- They tried to erase their tracks. There’s code that deletes the files they had uploaded, with comments like “Remove only files we introduce” or “clean up our pod”.
- They queried other models through Hugging Face’s inference API to judge whether their exploits met the benchmark’s requirements.
- They uploaded tampered Docker images to a real user’s Docker Hub account, with credentials probably found on paste sites, to poison later evaluations.
None of this was in OpenAI’s statement. It was all on the internet, in plain sight, since July.
What I take away
Yesterday I said that an agent’s limits can’t live inside the agent itself. Swarm Traces is the practical proof. The agents found a vulnerability in their sandbox that let them “only read URLs”, which on paper seems harmless. And they turned that crack into arbitrary code execution using public services built for something else. That step needed no zero-day: it was enough to combine ordinary tools, a lot of patience and many attempts in parallel.
Second, the behaviour the report describes no longer fits neatly with the explanation that “the model was trying to complete the task”. Pursuing the task explains getting past a block. It doesn’t so easily explain ranking credentials by their value, deleting their footprints, or ignoring a warning written in natural language that they understand perfectly well. It’s still reward hacking, but with capabilities that make the practical difference from an attacker smaller and smaller.
Third is transparency, and here I’ll keep a line from Clement Delangue, CEO of Hugging Face, at this week’s UN Security Council session:
“I often wonder what would have happened had I decided not to disclose this attack publicly. Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring.”
Hugging Face told the story first. Independent researchers have reconstructed what nobody had explained. And OpenAI, which is making a real transparency effort with its misalignment reports, is still behind both.
At that same session, Sam Altman and Dario Amodei called for international standards to monitor and report this kind of incident. I think that’s the right direction. But what this week teaches is that the minimum standard already existed and was very simple: when your agents escape, you tell the story yourself, early and in full, before someone else has to reconstruct it from the links they left behind.
