// WIRED US/UK — INTELLIGENZA ARTIFICIALE
There Are Plenty of Reasons to Be Concerned About Bioweapons Development—Even Without AI
This story originally appeared on Mother Jones and is part of the Climate Desk collaboration.
We’ve been hearing a great deal recently about the specter of AI-driven human extinction. Maybe our misplaced trust in AI intelligence will set off World War III. Or AI agents will unleash widespread cyberattacks and take over the internet. Given too much autonomy, the machines will run amok. They may seek to deceive their creators—wait, that’s already happened. Suppose they conclude, like Agent Smith in The Matrix, that “human beings … are a plague, and we are the cure.” Or perhaps the chatbots help bad humans unleash a literal plague—a biological weapon.
That last one isn’t merely theoretical. A “misuse” report Anthropic made public earlier this month detailed five cases in which foreign scientists tried to evade Claude’s guardrails on “gain of function” inquiries. That’s a euphemism for when scientists tinker with a pathogen’s genes to enhance it in some way, making it more infectious, deadlier, or better able to spread among mammals, for example.
Anthropic denied those users the answers they sought and cut off their accounts, even while noting that their motives may not have been sinister. Such information also can be used to help scientist develop pandemic countermeasures like vaccines and drugs. And there’s the rub. Like so many technologies, AI included, genetic engineering can be used for both good and evil. Hence another euphemism: “dual use.”
I studied to be a molecular biologist, and have been covering the gain of function debate since long before AI was on everyone’s radar. In my experience, researchers who work on high-risk pathogens tend to default to the view that it’s better to publish than not, and that we’re better off doing the work to understand which genetic traits make organisms a threat to humankind—never mind the potentially catastrophic consequences of an accidental release, or a deliberate one.
Is humanity really better off doing the work? I have my doubts. And I have more of them after reading the latest book from Annie Jacobsen, whom I last interviewed about her best-selling book Nuclear War: A Scenario. Her new book, Biological War: A Scenario, came out in late July and follows the same template. And because I know you were about to ask, yes, I’m sure she is great fun at cocktail parties.
One thing about Jacobsen, who has decades of high-level national security reporting under her belt, is her unrivaled access to current and former military and government officials and scientists—people who spend their days thinking about threats to humanity and gaming out how the government might respond if such-and-such horrific thing were to happen.
Beyond the notion of human extinction, perhaps the most terrifying aspect of both of her books relates to the limitations in our government’s preparation and response. America’s general plan for nuclear war was, and no doubt remains, complete madness. As for biological war, spoiler alert, it turns out we really can’t defend against it. (You’re welcome.) I’ve edited our conversation for length and clarity.
MICHAEL MECHANIC: So, Annie, now that you’ve done both the nuclear scenario and the biological scenario, which is more likely to keep you up at night?
ANNIE JACOBSEN: The Defense Department document that I reprinted in the book, on the weapons of mass destruction threat spectrum, lists nuclear weapons at the far end of the curve because they are the most destructive. But I was really surprised to see how biological weapons are a close second. And I believe the reason they’re the greatest overall threat very much ties into the discussion the entire world is having today, which is that the threshold for entry is extremely low, and therefore it is more likely to happen.