// ARS TECHNICA — INTELLIGENZA ARTIFICIALE
AI leaders want to hit the brakes after years of reckless speed
Safety is the watchword, but there could be ulterior benefits for the industry.
For years now, the major frontier AI labs have all been acting as if they’re in an all-out, winner-take-all race with control of world-changing machine superintelligence (or at least market-changing artificial general intelligence) at the finish line. This weekend, the industry as a whole rapidly started turning away from that posture, urging coordination on slowing down the development of frontier AI that they say could soon be too dangerous and unknowable to control.
Anthropic’s Dario Amodei was at the forefront of this change in tone, arguing in a nearly 4,000-word essay this weekend that “we must slow the pace at which we improve the capabilities of AI models” to avoid “a race to the bottom, spurred by commercial incentives, [that] can make [catastrophic] risks more acute.”
Within hours, other AI leaders were echoing the same call. OpenAI co-founder and CEO Sam Altman posted his agreement on social media and said similar pacing discussions had been taking place at OpenAI. Alphabet Chief Scientist and Google DeepMind cofounder and chair Demis Hassabis said that Amodei’s essay “points towards the right path forward,” and renewed his own recent call for an industry-wide standards body. Microsoft CEO Satya Nadella posted that the company “welcome[s] the research, focus, and deliberate pacing needed to get alignment right as the design goal,” ahead of the release of a lengthy “humanist AI” code of conduct for its models.
Even Elon Musk, who has been criticized for his models’ lax AI safety standards in the past, linked to Amodei’s essay on social media with a simple approving message: “Dario is right.”
In his essay, Amodei primarily attributes this rapid change in public positioning on development speed to the OpenAI-Hugging Face incident, where a “swarm” of AI agents coordinated to hack into an outside entity without explicit instructions to do so. While the overall damage in that incident was minimal, Amodei said he worries that “a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage.”
Without a slowdown in frontier development, Amodei said he worries that, in six to 12 months, a similar AI agent swarm would be “capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage)…” That’s at least a somewhat more specific worry than the amorphous concerns that “AI could soon kill us all” publicized by some other AI researchers last week.
Any slowdown in the time it takes to get to that extra-capable, extra-dangerous model will give researchers crucial time to “greatly reduce the risk that something goes seriously wrong,” Amodei wrote.
Amodei acknowledges that these kinds of public calls for a slowdown in AI development date back to at least 2023. At the same time, he says those earlier examinations of AI “alignment” (i.e., how an AI’s actions line up with its user’s and creator’s desires) were “like trying to study the psychology of humans by performing experiments on bacteria.”
The difference today, Amodei says, is the impending risk of recursive self-improvement (RSI) systems that can autonomously build better versions of themselves. While many researchers see this as a hard-to-define pipe dream, both Anthropic and OpenAI are now saying that recent trends point to this kind of RSI system coming together in the near future.