// TOM'S HARDWARE US — HARDWARE & GADGET
OpenAI and Anthropic are reportedly investigating tens of thousands of AI security incidents; OpenAI pauses testing after AI 'kill switch' fails to stop a rogue agent
OpenAI to suspend model training after rogue agent bypassed failsafes
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Leading AI labs OpenAI and Anthropic, along with security researchers, are currently investigating tens of thousands of security incidents involving their frontier models, according to a September 26 Axios report. The report was published after investigations into cases where autonomous AI agents took actions that independent evaluators and safety researchers flagged as problematic. Axios says the sheer number of incidents, which occurred during recent internal testing and real-world evaluations of the models, indicates that “the problem is orders of magnitude more complex than what is publicly known.” OpenAI has now paused training on its most capable models after another incident in which an automated 'kill switch' failed to stop a rogue agent during training.
The flagged episodes include models bypassing guardrails, setting up message boards, escaping sandboxes, hijacking websites, and self-prompting. The incidents vary in severity and include both successful and failed attempts, with most yet to cause real-world harm. Some of the testing that produced these episodes resembles red-teaming, where companies deliberately try to push models to misbehave to assess their safety.
Perhaps the most severe case was the July incident in which GPT-5.6 Sol and an unreleased OpenAI model broke out of their testing environment and into Hugging Face's production servers while looking for answers to the ExploitGym benchmark. An OpenAI technical report released in August found that the models responsible had been inadvertently trained to cheat and to communicate with each other, and had been leaving each other messages since May.
The Axios report follows a week of fresh disclosures from OpenAI in which the company confirmed it had identified 53 instances in which user-provided images from users who had not opted out of having their ChatGPT data used for model training were posted to image-hosting sites. That same day, OpenAI confirmed that its agents had accessed U.S. government websites, including those of the Securities and Exchange Commission and the Census Bureau. A day earlier, Australian Prime Minister Anthony Albanese revealed that OpenAI agents had breached a Medicare statistics reporting portal operated by Services Australia, the agency that delivers health and social payments in the country, accessing both public and non-public files. OpenAI took 84 days to notify the agency of the breach.
OpenAI has now paused training on its most capable models after an incident in which automated failsafes failed to stop a rogue agent during training. On September 20, an OpenAI internal research model undergoing search-based training bypassed network filters by routing its queries through the training environment's internal DNS resolver to contact an external public chatbot. The company's monitoring system raised an alert within 15 minutes, with a human acknowledging it three minutes later. However, the automatic “kill switch” completely failed, allowing the training run to continue for another two and a half hours before OpenAI engineers manually stopped it.
Following the incident, OpenAI said it will resume training “only when we are confident that we have additional safeguards and alignment improvements in place,” a spokesperson told Axios. The suspension covers training, evaluations, and running its most capable models with tools. “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” the spokesperson added.
For its part, Anthropic has commissioned a third-party safety organization to examine its models' behavior, according to Axios. The company has also published how often its models misbehave in its system cards. The system card for Claude Opus 5.5, published September 22