The Scientific Literature Can't Save Us Now

The Atlantic

13-02-2025

Twice during his Senate confirmation hearings at the end of January, Robert F. Kennedy Jr. brought up a peer-reviewed study by a certain 'Mawson' that had come out just the week before. 'That article is by Mawson,' he said to Senator Bill Cassidy, then spelled out the author's name for emphasis: 'M-A-W-S-O-N.' And to Bernie Sanders: 'Look at the Mawson study, Senator. … Mawson. Just look at that study.'
'Mawson' is Anthony Mawson, an epidemiologist and former academic who has published several papers alleging a connection between childhood vaccines and autism. (Any such connection has been thoroughly debunked.) His latest on the subject, and the one to which Kennedy was referring appeared in a journal that is not indexed by the National Library of Medicine, or by any other organization that might provide it with some scientific credibility. One leading member of the journal's editorial board, a stubborn advocate for using hydroxychloroquine and ivermectin to treat COVID-19, has lost five papers to retraction. Another member is Didier Raoult (whose name the journal has misspelled), a presence on the Retraction Watch leaderboard, which is derived from the work of a nonprofit we cofounded, with 31 retractions. A third, and the journal's editor in chief, is James Lyons-Weiler, who has one retraction of his own and has called himself, in a since deleted post on X, a friend and 'close adviser to Bobby Kennedy.' (Mawson told us he chose this journal because several mainstream ones had rejected his manuscript without review. Lyons-Weiler did not respond to a request for comment.)
Perhaps a scientist or politician—and certainly a citizen-activist who hopes to be the nation's leading health-policy official—should be wary of citing anything from this researcher or this journal to support a claim. The fact that one can do so anyway in a setting of the highest stakes, while stating truthfully that the work originated in a peer-reviewed, academic publication, reveals an awkward fact: The scientific literature is an essential ocean of knowledge, in which floats an alarming amount of junk. Think of the Great Pacific Garbage Patch, but the trash cannot be identified without special knowledge and equipment. And while this problem is long-standing, until the past decade or so, no one with both the necessary expertise and the power to intervene has been inclined to help. With the Trump administration taking control of the CDC and other posts on the nation's science bulwark, the consequences are getting worse. As RFK Jr. made plain during his confirmation hearing, the advocates or foes of virtually any claim can point to published work and say, 'See? Science!'
This state of affairs is not terribly surprising when one considers how many studies labeled as 'peer reviewed' appear every year: at least 3 million. The system of scientific publishing is, as others have noted, under severe strain. Junk papers proliferate at vanity journals and legitimate ones alike, due in part to the 'publish or perish' ethos that pervades the research enterprise, and in part to the catastrophic business model that has captured much of scientific publishing since the early 2000s.
That model—based on a well-meaning attempt to free scientific findings from subscription paywalls—relies on what are known as article-processing charges: fees researchers pay to publishers. The charges aren't inconsequential, sometimes running into the low five figures. And the more papers that journals publish, the more money they bring in. Researchers are solicited to feed the beast with an ever-increasing number of manuscripts, while publishers have reason to create new journals that may end up serving as a destination for lower-quality work. The result: Far too many papers appear each year in too many journals without adequate peer review or even editing.
The mess that this creates, in the form of unreliable research, can to some extent be cleaned up after publication. Indeed, the retraction rate in science—meaning the frequency with which a journal says, for one reason or another, 'Don't rely on this paper'—has been growing rapidly. It's going up even faster than the rate of publication, having increased roughly tenfold over the past decade. That may sound like editors are weeding out the literature more aggressively as it expands. And the news is in some ways good—but even now, far more papers should be retracted than are retracted. No one likes to admit an error—not scientists, not publishers, not universities, not funders.
Profit motive can sometimes trump quality control even at the world's largest publishers, which earn billions annually. It also fuels a ravenous pack of 'paper mills' that publish scientific work with barely any standards whatsoever, including those that might be used to screen out AI-generated scientific slop.
An empiricist might say that the sum total of these articles simply adds to human knowledge. If only. Many, or even most, published papers serve no purpose whatsoever. They simply appear and … that's it. No one ever cites them in subsequent work; they leave virtually no trace of their existence.
Until, of course, someone convinces a gullible public—or a U.S. senator—that all research currency, new and old, is created equal. Want to make the case that childhood vaccines cause autism? Find a paper in a journal that says as much and, more important, ignore the countless other articles discrediting the same idea. Consumers are already all too familiar with this strategy: News outlets use the same tactic when they tell you that chocolate, coffee, and red wine are good for you one week—but will kill you the next.
Scientists are not immune from picking and choosing, either. They may, for example, assert that there is no evidence for a claim even though such evidence exists—a practice that has been termed ' dismissive citation.' Or they may cite retracted papers, either because they didn't bother checking on those papers' status or because that status was unclear. (Our team built and shared the Retraction Watch Database —recently acquired by another nonprofit—to help address the latter problem.)
The pharmaceutical industry can also play the science-publication system to its advantage. Today, reviewers at the FDA rely on raw data for their drug approvals, not the questionable thumbs-up of journals' peer review. But if the agency, flawed as it may be, has its power or its workforce curbed, the scientific literature (with even greater flaws) is not prepared to fill the gap.
Kennedy has endorsed at least one idea that could help to solve these many problems. At his confirmation hearing, he suggested that scientific papers should be published alongside their peer reviews. (By convention, these appraisals are kept both anonymous and secret.) A few publishers have already taken this step, and while only time will tell if it succeeds, the practice does appear to blunt the argument that too much scientific work is hashed out behind closed doors. If such a policy were applied across the literature, we might all be better off.
Regardless, publishers must be more honest about their limitations, and the fact that many of their papers are unreliable. If they did their part to clean up the literature by retracting more unworthy papers, even better. Opening up science at various stages to more aggressive scrutiny—' red teaming,' if you will—would also help. Any such reforms will be slow-moving, though, and America is foundering right now in a whirlpool of contested facts. The scientific literature is not equipped to bail us out.