“AI is the future of peer review”

AI Ready to Revolutionize Scientific Gatekeeping as Peer Review Crisis Looms

The crushing weight of millions of academic papers awaiting expert evaluation could soon push the scientific community toward an AI-assisted future, according to a prominent medical journal editor.

In an editorial published this week in Critical Care Medicine, a former top editor at one of medicine’s most prestigious journals argues that artificial intelligence should become an integral part of how scientific papers are evaluated before publication.

“Peer review at biomedical journals has been essentially unchanged for many decades. Although compensating peer reviewers would likely help to receive timely reviews, it is probably not feasible on a wide scale. In addition, peer review has well-known limitations,” said Howard Bauchner, MD, professor of pediatrics at Boston University Chobanian & Avedisian School of Medicine and former editor-in-chief of the Journal of the American Medical Association.

The scale of the challenge is staggering. This year alone, approximately three million articles will be indexed in major scientific databases, with each requiring multiple expert evaluations. Add in the papers that are reviewed but ultimately rejected, and the academic community will need to perform roughly 10 million peer reviews in 2025 – a number that continues to grow as biomedical research expands globally.

For scientists juggling research, teaching, and administrative duties, the unpaid labor of thorough peer review often becomes an afterthought. Many rush through evaluations or decline review requests entirely, creating bottlenecks in scientific publishing that can delay important findings from reaching the public.

Bauchner’s proposed solution is straightforward: “We believe peer review should include some form of initial review by AI, assisting editors in decisions on which articles to send out for external peer review.”

The editorial tackles one of academia’s uncomfortable truths – human reviewers bring their own biases to the evaluation process. Bauchner references a major study comparing review approaches, noting that “when reviewers were aware of the authors’ identity (single-blind), they gave a more favorable rating from countries with higher English proficiency and higher income.”

While acknowledging that AI could also be biased, Bauchner suggests that “models could be taught to disregard who the authors are and where they come from,” potentially offering a more objective initial screening process.

The idea isn’t merely theoretical. Several independent companies already offer AI review services to authors before they submit manuscripts to journals. According to one study cited by Bauchner, authors found feedback from GPT-4 “to be more helpful than feedback from some peer reviewers.”

Beyond addressing bias, AI could enforce standards that human reviewers frequently overlook. Bauchner notes that while journals often request authors follow specific reporting guidelines, there’s “no evidence that peer reviewers actually check adherence to these guidelines” – a tedious but critical task that AI could perform consistently.

Perhaps most compellingly, AI may prove better at detecting potential research fraud, a growing concern as publication pressure intensifies across academia.

The transition toward AI-assisted peer review faces obstacles, including resistance from traditionalists and concerns about surrendering human judgment to algorithms. However, as the avalanche of papers continues to outpace human capacity, Bauchner’s conclusion is resolute: “As it continues to improve, it is time to embrace a different approach, an approach that is likely to be more efficient and more effective—review by AI.”

With spring conference season approaching and thousands of new manuscripts entering the submission pipeline, the academic community may soon need to decide whether to hold fast to tradition or welcome AI as the newest member of the editorial team.