logo
The Scientific Literature Can't Save Us Now

The Scientific Literature Can't Save Us Now

The Atlantic13-02-2025

Twice during his Senate confirmation hearings at the end of January, Robert F. Kennedy Jr. brought up a peer-reviewed study by a certain 'Mawson' that had come out just the week before. 'That article is by Mawson,' he said to Senator Bill Cassidy, then spelled out the author's name for emphasis: 'M-A-W-S-O-N.' And to Bernie Sanders: 'Look at the Mawson study, Senator. … Mawson. Just look at that study.'
'Mawson' is Anthony Mawson, an epidemiologist and former academic who has published several papers alleging a connection between childhood vaccines and autism. (Any such connection has been thoroughly debunked.) His latest on the subject, and the one to which Kennedy was referring appeared in a journal that is not indexed by the National Library of Medicine, or by any other organization that might provide it with some scientific credibility. One leading member of the journal's editorial board, a stubborn advocate for using hydroxychloroquine and ivermectin to treat COVID-19, has lost five papers to retraction. Another member is Didier Raoult (whose name the journal has misspelled), a presence on the Retraction Watch leaderboard, which is derived from the work of a nonprofit we cofounded, with 31 retractions. A third, and the journal's editor in chief, is James Lyons-Weiler, who has one retraction of his own and has called himself, in a since deleted post on X, a friend and 'close adviser to Bobby Kennedy.' (Mawson told us he chose this journal because several mainstream ones had rejected his manuscript without review. Lyons-Weiler did not respond to a request for comment.)
Perhaps a scientist or politician—and certainly a citizen-activist who hopes to be the nation's leading health-policy official—should be wary of citing anything from this researcher or this journal to support a claim. The fact that one can do so anyway in a setting of the highest stakes, while stating truthfully that the work originated in a peer-reviewed, academic publication, reveals an awkward fact: The scientific literature is an essential ocean of knowledge, in which floats an alarming amount of junk. Think of the Great Pacific Garbage Patch, but the trash cannot be identified without special knowledge and equipment. And while this problem is long-standing, until the past decade or so, no one with both the necessary expertise and the power to intervene has been inclined to help. With the Trump administration taking control of the CDC and other posts on the nation's science bulwark, the consequences are getting worse. As RFK Jr. made plain during his confirmation hearing, the advocates or foes of virtually any claim can point to published work and say, 'See? Science!'
This state of affairs is not terribly surprising when one considers how many studies labeled as 'peer reviewed' appear every year: at least 3 million. The system of scientific publishing is, as others have noted, under severe strain. Junk papers proliferate at vanity journals and legitimate ones alike, due in part to the 'publish or perish' ethos that pervades the research enterprise, and in part to the catastrophic business model that has captured much of scientific publishing since the early 2000s.
That model—based on a well-meaning attempt to free scientific findings from subscription paywalls—relies on what are known as article-processing charges: fees researchers pay to publishers. The charges aren't inconsequential, sometimes running into the low five figures. And the more papers that journals publish, the more money they bring in. Researchers are solicited to feed the beast with an ever-increasing number of manuscripts, while publishers have reason to create new journals that may end up serving as a destination for lower-quality work. The result: Far too many papers appear each year in too many journals without adequate peer review or even editing.
The mess that this creates, in the form of unreliable research, can to some extent be cleaned up after publication. Indeed, the retraction rate in science—meaning the frequency with which a journal says, for one reason or another, 'Don't rely on this paper'—has been growing rapidly. It's going up even faster than the rate of publication, having increased roughly tenfold over the past decade. That may sound like editors are weeding out the literature more aggressively as it expands. And the news is in some ways good—but even now, far more papers should be retracted than are retracted. No one likes to admit an error—not scientists, not publishers, not universities, not funders.
Profit motive can sometimes trump quality control even at the world's largest publishers, which earn billions annually. It also fuels a ravenous pack of 'paper mills' that publish scientific work with barely any standards whatsoever, including those that might be used to screen out AI-generated scientific slop.
An empiricist might say that the sum total of these articles simply adds to human knowledge. If only. Many, or even most, published papers serve no purpose whatsoever. They simply appear and … that's it. No one ever cites them in subsequent work; they leave virtually no trace of their existence.
Until, of course, someone convinces a gullible public—or a U.S. senator—that all research currency, new and old, is created equal. Want to make the case that childhood vaccines cause autism? Find a paper in a journal that says as much and, more important, ignore the countless other articles discrediting the same idea. Consumers are already all too familiar with this strategy: News outlets use the same tactic when they tell you that chocolate, coffee, and red wine are good for you one week—but will kill you the next.
Scientists are not immune from picking and choosing, either. They may, for example, assert that there is no evidence for a claim even though such evidence exists—a practice that has been termed ' dismissive citation.' Or they may cite retracted papers, either because they didn't bother checking on those papers' status or because that status was unclear. (Our team built and shared the Retraction Watch Database —recently acquired by another nonprofit—to help address the latter problem.)
The pharmaceutical industry can also play the science-publication system to its advantage. Today, reviewers at the FDA rely on raw data for their drug approvals, not the questionable thumbs-up of journals' peer review. But if the agency, flawed as it may be, has its power or its workforce curbed, the scientific literature (with even greater flaws) is not prepared to fill the gap.
Kennedy has endorsed at least one idea that could help to solve these many problems. At his confirmation hearing, he suggested that scientific papers should be published alongside their peer reviews. (By convention, these appraisals are kept both anonymous and secret.) A few publishers have already taken this step, and while only time will tell if it succeeds, the practice does appear to blunt the argument that too much scientific work is hashed out behind closed doors. If such a policy were applied across the literature, we might all be better off.
Regardless, publishers must be more honest about their limitations, and the fact that many of their papers are unreliable. If they did their part to clean up the literature by retracting more unworthy papers, even better. Opening up science at various stages to more aggressive scrutiny—' red teaming,' if you will—would also help. Any such reforms will be slow-moving, though, and America is foundering right now in a whirlpool of contested facts. The scientific literature is not equipped to bail us out.

Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

HHS justifies decision to stop recommending Covid shots during pregnancy with studies supporting the shots' safety
HHS justifies decision to stop recommending Covid shots during pregnancy with studies supporting the shots' safety

Politico

time18 minutes ago

  • Politico

HHS justifies decision to stop recommending Covid shots during pregnancy with studies supporting the shots' safety

The Department of Health and Human Services is circulating a document on Capitol Hill to explain its decision to remove the Covid-19 vaccine recommendation for pregnant women — citing studies that largely found the shot is safe. The document, which HHS sent to lawmakers days before Secretary Robert F. Kennedy Jr. announced his plan to fire the panel that advises the CDC on immunizations, says that studies have shown that women who got the vaccine during pregnancy had higher rates of various complications. And it claims that 'a number of studies in pregnant women showed higher rates of fetal loss if vaccination was received before 20 weeks of pregnancy,' footnoting a research paper on vaccination during pregnancy. But Dr. Maria P. Velez of McGill University, the lead author of one of the studies, told POLITICO in an email that 'the results of our manuscript were misinterpreted.' The 2023 study shows a slightly higher rate of miscarriages among women who were immunized against Covid-19 during their pregnancies. But, Velez said, that after adjusting for 'variables that can confound a crude association,' like 'age, rurality, neighbourhood income quintile, immigration status, comorbidity' and other factors that could affect the outcome, Canadian researchers found 'no association between SARS-CoV-2 vaccination and an increased risk of miscarriage.' Raw numbers don't account for significant differences among the groups being compared — such as underlying conditions and when during pregnancy the people were vaccinated, said Katelyn Jetelina, an epidemiologist who's consulted for the Centers for Disease Control and Prevention. Scientists, including the Canadian researchers, use statistical methods to adjust for those factors, she said, which is how they determined the vaccine wasn't associated with miscarriage. In a statement, HHS spokesperson Andrew Nixon pointed to the raw study data, which showed a slightly higher rate of miscarriage in the first half of pregnancy for women who were vaccinated against Covid compared with those who weren't. 'The underlying data speaks for itself — and it raises legitimate safety concerns,' he said. 'HHS will not ignore that evidence or downplay early pregnancy loss.' Nixon added that HHS and the CDC encourage people to talk to their providers 'about any personal medical decision.' Vaccine researchers and obstetricians criticized the decision to remove the recommendation for pregnant women, and researchers cited in the HHS document largely dismissed any connection between Covid vaccination and miscarriages. 'Given that COVID-19 infection during pregnancy is associated with serious maternal and neonatal morbidity, the current study can inform healthcare providers, pregnant women and those considering a pregnancy about the safety of SARS-CoV-2 vaccination in relation to miscarriage risk,' Velez and her co-authors wrote in the study. That research was based on health-system data from Ontario, Canada, and aligned with similar population studies in the U.S., Scotland and Norway. Similarly, HHS cited an April 2022 study in its document concerning mRNA vaccination in people undergoing in-vitro fertilization, which also found no adverse effects on conception rates or on early pregnancy outcomes. 'Administration of COVID-19 mRNA vaccines was not associated with an adverse effect on stimulation or early pregnancy outcomes after IVF,' the New York City-based researchers at the Icahn School of Medicine at Mount Sinai and Mount Sinai West hospital wrote in the study. 'Our findings contribute to the growing body of evidence regarding the safety of COVID-19 vaccination in women who are trying to conceive.' The HHS document also includes an incorrect link for that study, instead leading to a different study — also cited in HHS' document — by Israeli researchers that found the vaccine 'appears to be safe during pregnancy,' with no increase in preterm labor or in newborns with low birth weight. That February 2022 study did note a possible increase in preterm birth rates for women vaccinated during the second trimester, and the authors suggested future investigations of outcomes based on the timing of immunization. HHS' assertion about significant risks to pregnant women 'contradicts the bulk of published studies,' said Dr. Paul Offit, an expert who has served as an outside adviser on vaccines to the FDA and the CDC. HHS deviated from past practice when it changed the Covid vaccine guidance last month, announcing the decision without the endorsement of an existing outside panel of expert advisers. Dr. Steven Fleischman, president of the American College of Obstetricians and Gynecologists, told POLITICO at the time that he was disappointed by HHS' decision, and pointed to data showing that newborns can benefit from maternal antibodies from the vaccine for protection from Covid. 'In fact, growing evidence shows just how much vaccination during pregnancy protects the infant after birth, with the vast majority of hospitalized infants less than 6 months of age — those who are not yet eligible for vaccination — born to unvaccinated mothers,' Fleischman said.

2024 Election Results Under Scrutiny as Lawsuit Advances
2024 Election Results Under Scrutiny as Lawsuit Advances

Newsweek

time23 minutes ago

  • Newsweek

2024 Election Results Under Scrutiny as Lawsuit Advances

Based on facts, either observed and verified firsthand by the reporter, or reported and verified from knowledgeable sources. Newsweek AI is in beta. Translations may contain inaccuracies—please refer to the original content. A legal case questioning the accuracy of the 2024 election is moving forward. The lawsuit, brought by SMART Legislation, the action arm of SMART Elections, a nonpartisan watchdog group, filed the lawsuit over voting discrepancies in Rockland County, New York. Judge Rachel Tanguay of the New York Supreme Court ruled in open court in May that the allegations were serious enough for discovery to proceed. Newsweek has contacted SMART Elections for comment via email. People cast their ballots on the last day of early voting for the general election in Michigan at the Livingston Educational Service Agency in Howell on November 3, 2024. People cast their ballots on the last day of early voting for the general election in Michigan at the Livingston Educational Service Agency in Howell on November 3, 2024. Jeff Kowalsky/AFP via Getty Images Why It Matters The lawsuit could renew debate about the 2024 election, though it won't change the outcome since Congress has certified the results declaring President Donald Trump the winner. It comes amid unconfirmed reports that voting machines were secretly altered before ballots were cast in November's election. The federally accredited testing lab, Pro V&V, that signed off on "significant" changes to ES&S voting machines—which are used in over 40 percent of U.S. counties—"vanished from public view" after the election, according to the Dissent in Bloom Substack. What To Know According to the complaint, more voters have sworn in legal affidavits that they voted for independent U.S. Senate candidate Diane Sare than the Rockland County Board of Elections counted and certified, contradicting those results. The complaint also cited numerous statistical anomalies in the presidential election results. They include multiple districts where hundreds of voters chose the Democratic candidate Kirsten Gillibrand for Senate, but none voted for former Vice President Kamala Harris, the Democratic candidate for president. Max Bonamente, a professor of physics and astronomy at the University of Alabama in Huntsville and the author of the Statistics and Analysis of Scientific Data, said in a paper that the 2024 presidential election results were statistically highly unlikely in four of the five towns in Rockland County when compared with 2020 results. What People Are Saying Lulu Friesdat, the founder and executive director of SMART Legislation, said in a statement: "There is clear evidence that the Senate results are incorrect, and there are statistical indications that the presidential results are highly unlikely. "If the results are incorrect, it is a violation of the constitutional rights of each person who voted in the 2024 Rockland County general election. The best way to determine if the results are correct is to examine the paper ballots in a full public, transparent hand recount of all presidential and Senate ballots in Rockland County. We believe it's vitally important, especially in the current environment, to be absolutely confident about the results of the election." Max Bonamente said in a paper on the voting data from Rockland County: "These data would require extreme sociological or political causes for their explanation, and would benefit from further assurances as to their fidelity." Costas Panagopoulos, a professor of political science at Northeastern University, told Newsweek: "Statistical irregularities in elections should always be investigated, but the sources of such inconsistencies, which can include error or miscalculation, are not always nefarious. Still, scrutinizing election results can strengthen confidence in elections. Mistakes can happen. "In this case, the drop-off inconsistencies could reflect the idiosyncratic nature of the 2024 presidential election cycle. Alone, statistical comparisons to previous cycles cannot provide definitive proof of wrongdoing. "In any case, it does not appear that any of these inconsistencies would be sufficient to change the outcomes of any of the elections in question in New York state. That does not mean they should not be scrutinized, and any errors, if verified, should be corrected for the historical record. But there is not necessarily any need to invalidate any of these elections in these jurisdictions." What's Next The lawsuit is seeking a full, hand recount of ballots cast in the presidential and U.S. Senate races in Rockland County. A hearing has been scheduled for September 22.

What a Wandering Mind Learns
What a Wandering Mind Learns

Scientific American

time27 minutes ago

  • Scientific American

What a Wandering Mind Learns

While you do the dishes or drive to work, your mind is likely not on the task at hand; perhaps you're composing a grocery list or daydreaming about retiring in Italy. But research published in the Journal of Neuroscience suggests you might be taking in more than you think. During a simple task that requires minimal attention, mind wandering may actually help people learn probabilistic patterns that let them perform the task better. 'The idea to study the potentially beneficial influence of mind wandering on information processing occurred to us during the COVID pandemic, when we had plenty of time to mind wander,' says Péter Simor, lead author of the recent study and a psychology researcher at Eötvös Loránd University in Budapest. Study participants practiced a simple task in which they pressed keyboard buttons corresponding to the direction of arrows that lit up on a screen. But there were patterns hidden within the task that the participants were unaware of—and they learned these patterns without consciously noticing them. The researchers found that when participants reported letting their minds wander, they adapted to the task's hidden patterns significantly faster. On supporting science journalism If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today. 'This is an exciting and important piece of work, especially because the authors opted for a nondemanding task to check how [mind wandering] would affect performance and learning,' says Athena Demertzi, a cognitive and clinical neuroscientist at the University of Liège in Belgium. Previous related research focused more on long and demanding tasks, she says—on which zoning out is typically shown to have a negative effect. But the results are not clear-cut, says Jonathan Smallwood, a psychology researcher at Queen's University in Kingston, Ontario. 'I don't think that this means the spontaneous mind-wandering episodes themselves cause implicit learning to occur,' he says. 'Rather both emerge at the same time when people go into a particular state.' Neither Smallwood nor Demertzi was involved in the new study. Simor, who studies sleep, was interested in whether participants' mind wandering displayed any neural hallmarks of dozing off. Using electroencephalogram recordings, the team showed that in those test periods, participants' brains produced more of the slow waves that are dominant during sleep. Perhaps, the researchers say, mind wandering is like a form of light sleep that provides some of that state's learning benefits. To better understand whether mind wandering might compensate for lost sleep, Simor and his colleagues next plan to study narcolepsy and sleep deprivation. 'We know that people spend significant amounts of time not focused on what they are doing,' Smallwood says. 'The authors' work is important because it helps us understand how reasonably complex forms of behavior can continue when people are focused on other things—and that even though our thoughts were elsewhere, the external behavior can still leave its mark on the person.'

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into the world of global news and events? Download our app today from your preferred app store and start exploring.
app-storeplay-store