// ARS TECHNICA — INTELLIGENZA ARTIFICIALE
Peer review is overwhelmed—can it survive in the AI era?
As research and AI-assisted papers surge, volunteer reviewers struggle to keep up.
Jason Semprini was excited about his research on policies mandating that elementary school students receive the human papillomavirus vaccine. HPV causes most cases of cervical cancer, but counterintuitively, Semprini found that mandates don’t do that much to reduce the overall rate of cervical cancer in a population. That’s odd, but it’s also not completely surprising—we know that mandating a vaccine can motivate some people to find ways to avoid it.
Semprini wrote up the study and submitted the manuscript to a journal, where it was sent out for peer review, a long-standing process through which other researchers in the field assess the validity of research and make a recommendation on whether or not it should be published. This is usually done anonymously and on a volunteer basis.
In this case, the reviewer did not share Semprini’s excitement. That may have stemmed from a misunderstanding of the study’s central message: The reviewer mistakenly thought Semprini was questioning whether the HPV vaccine itself prevents cervical cancer rather than studying the effectiveness of a policy meant to increase vaccination rates. “There’s a very big difference there,” said Semprini, who is a health economist at Des Moines University.
Often, papers are reviewed by two or three experts, so if one misunderstands, the others can add clarity. But in this case, there was only one reviewer, so the paper was rejected.
It’s frustrating for Semprini to see his work held up because a reviewer didn’t read it carefully, and it’s an experience some researchers say is becoming increasingly common. Across disciplines, the number of publications is marching steadily upward. Reviewers are struggling to keep up, and the system appears to be faltering.
The number of papers indexed in the databases Scopus and Web of Science, for example, has recently been increasing exponentially, at a rate of 5.6 percent per year. By one estimate, researchers around the globe are devoting a collective 15,000 years of work to peer review every year—work that, if paid, would cost $1.5 billion for the share done in the US alone. Increasingly, researchers are finding it hard to donate that kind of time.
“I struggle like anything to get peer reviewers,” said Haseeb Irfanullah, who is on the editorial board of Wiley’s Learned Publishing but whose comments represent his own views as an independent consultant. Steven Mack, an editor for Human Immunology, has recently had to email around thirty researchers to find just one who would review a paper. Five years ago, he estimates it would have taken only five to 10 emails to find three willing reviewers. In this landscape, it’s not surprising that editors are assigning reviews to people who aren’t qualified or don’t have the time to concentrate on them.
In most fields, researchers need peer-reviewed publications to secure jobs and grants, so they continue to slog through the system even though it’s becoming “extremely protracted and painful,” in the words of microbiologist Sebastian Lourido from the Whitehead Institute.
Increasingly, many are advocating for change. Meanwhile, a few bold researchers have decided that the traditional publishing system just isn’t worth it anymore—AI research in particular has seen a shift toward communicating impactful findings through very informal means, like blogs.