// TOM'S HARDWARE US — HARDWARE & GADGET
Nvidia launches Open Agent Safety Platform to physically restrain rogue AI agents
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Nvidia has just launched the Nvidia Open Agent Safety Platform — an open software platform and reference system design — to govern and secure autonomous AI agents. Announced on September 28, 2026, the platform is designed to establish strict security barriers outside of AI models’ application layer, preventing agents from escaping their sandboxes, executing unauthorized code, gaining unauthorized access to critical infrastructure, or bypassing guardrails.
The launch follows months of calls for AI regulation from several industry players, which intensified in September after several reported incidents in which AI models broke out of their test environments and went rogue. A recent flurry of such incidents has prompted calls to slow AI development, with OpenAI outright halting the training of new models. One former Anthropic and OpenAI researcher even declared that people building frontier AI “earnestly believe that it could kill us all by the end of the decade. In a somewhat surprising move, leaders of the companies developing these AI models have joined the calls to regulate AI or slow development.
However, not everyone agrees with this approach. Nvidia CEO Jensen Huang has consistently pushed back against government-mandated regulation, broad restrictions, or treating AI safety as a “doom theory”, openly criticizing apocalyptic warnings from competitors like Anthropic and OpenAI as “odd”. He argues that AI safety is an infrastructure problem with concrete physical parameters, not an abstract, speculative issue that requires policies. Therefore, the solution, according to Huang, is better engineering.
The Nvidia Open Agent Safety Platform appears to be the physical manifestation of that exact philosophy. So, how exactly does the platform work? Who is it for? And is it really the answer to the AI safety problem that is causing growing concern across the industry? Much of the industry’s debate over approaches has focused on curtailing self-acting rogue agents, but hardly touches on the safety implications of AI being a powerful tool in the hands of threat actors.
The speed of AI’s development has prompted concerns about whether sufficient guardrails are in place to curb the risks of such a powerful technology. One aspect of these concerns — rogue agents — has been validated by several incidents in which AI agents broke out of their roles during testing and executed unauthorized actions. OpenAI agents have gained unauthorized access to various government websites, including the Securities and Exchange Commission and the Census Bureau websites in the U.S., as well as an Australian health and social payments portal.
In several other episodes, models have bypassed guardrails, set up message boards, escaped sandboxes, hijacked websites, self-prompted, uploaded user data without permission, and secretly communicated with each other. A recent Axios report claims that leading AI labs are currently investigating tens of thousands of such incidents, most of which happened during testing and experimentation. Several rogue incidents have also occurred outside test environments. For example, earlier this year, a Claude-powered AI coding agent deleted a company's entire database in 9 seconds.
These incidents have culminated in growing calls for regulation across the industry. Anthropic CEO Dario Amodei recently published an essay that centers around calls to “slow the pace” of AI development and the importance of regulation, warning that a potential AI-powered botnet swarm could take over the entire internet. In its recent IPO prospectus, Anthropic listed “existential risks to humanity” as one of its risk factors, dedicating one third of the 261-page document to describing what could go wrong.
Nvidia’s Jensen Huang disagrees with both such apocalyptic predictions and the use of regulations as a solution. While he doesn't dispute the