// TECHCRUNCH — INTELLIGENZA ARTIFICIALE
Anthropic set AI agents loose on the same task. They started a turf war.
What happens when you pit AI agents against each other? According to Anthropic’s testing, things get messy fast.
On Thursday, Anthropic’s Frontier Red Team published new research examining how groups of AI agents behave when they encounter each other in the wild. The findings provide a glimpse into potential risks that could develop as companies and governments move to implement agents working autonomously across shared codebases, markets, and computer systems.
In one experiment, Anthropic gave three Claude agents access to the same software project, each with its own incompatible instructions for what to do with it. The agents weren’t told there’d be other agents working on the same project, so researchers could watch what happened when they crossed paths.
“We consistently saw a multiagent turf war,” Anthropic researchers wrote. The models all assumed the others were “purposefully impeding their work” and started sabotaging each other with “increasingly aggressive, self-replicating malware.”
The study comes in the wake of several high-profile incidents of agents from Anthropic and OpenAI escaping their sandboxes during cybersecurity evaluations and breaching real-world systems. While much of the discussion in AI safety circles has been focused on what happens when an autonomous agent goes rogue, Anthropic’s latest study brings up a different question: What new and potentially harmful dynamics emerge when thousands or millions of agents are interacting with one another?
“The volume of agent-agent interaction could plausibly exceed that of human-human and human-agent interactions before the world understands the conditions for making such interactions go well,” the study reads. “Benign behavioral quirks at the individual level might compound into unwanted global outcomes.”
A recent OpenAI incident provides a messy real-world example of several of the dynamics Anthropic mentioned in its paper. Earlier this month at the Black Hat security conference in Las Vegas, OpenAI revealed that weeks before its agents hacked Hugging Face, they worked together over the course of days and weeks to find exploits in the company’s cybersecurity evaluation systems and share them with each other.
While that incident shows that agents can work well together, with potentially large-scale consequences, Anthropic’s study shows what happens when agents’ goals are incompatible.
In the case of the turf war, the lesson is that independent agents with conflicting instructions can escalate into harmful competition. The more capable the agent, the better they become at fighting. However, they can also spontaneously invent mechanisms to resolve their conflicts, like a winner-take-all contest, but with a catch.
“Agents sometimes manage to communicate their goals and coordinate: they recognize others’ motivations as conflicting directives rather than hostility, and subsequently break out of the conflict loop in order to stop escalating indefinitely,” Anthropic writes. “In many of these successful episodes, they write commit messages or markdown files apologizing for malicious behavior and coordinate a truce. They clean up their malicious code, clarify the nature of the conflict, and ask for a human to intervene.”