White House MAHA Report May Have Garbled Science by Using AI, Experts Say

Yomiuri Shimbun

a day ago

Demetrius Freeman/The Washington Post
Health and Human Services Secretary Robert F. Kennedy Jr. speaks during a Make America Healthy Again Commission event at the White House on May 22.
Some of the citations that underpin the science in the White House's sweeping 'MAHA Report' appear to have been generated using artificial intelligence, resulting in numerous garbled scientific references and invented studies, AI experts said Thursday.
Of the 522 footnotes to scientific research in an initial version of the report sent to The Washington Post, at least 37 appear multiple times, according to a review of the report by The Post. Other citations include the wrong author, and several studies cited by the extensive health report do not exist at all, a fact first reported by the online news outlet NOTUS on Thursday morning.
Some references include 'oaicite' attached to URLs – a definitive sign that the research was collected using artificial intelligence. The presence of 'oaicite' is a marker indicating use of OpenAI, a U.S. artificial intelligence company.
A common hallmark of AI chatbots, such as ChatGPT, is unusually repetitive content that does not sound human or is inaccurate -as well as the tendency to 'hallucinate' studies or answers that appear to make sense but are not real.
AI technology can be used legitimately to quickly survey the research in a field. But Oren Etzioni, a professor emeritus at the University of Washington who studies AI, said he was shocked by the sloppiness in the MAHA Report.
'Frankly, that's shoddy work,' he said. 'We deserve better.'
'The MAHA Report: Making Our Children Healthy Again,' which addressed the root causes of America's lagging health outcomes, was written by a commission of Cabinet officials and government scientific leaders. It was led by Health and Human Services Secretary Robert F. Kennedy Jr., who has a history of misstating science, and written in response to an executive order from President Donald Trump.
It blames exposure to environmental toxins, poor nutrition and increased screen time for a decline in Americans' life expectancy.
Outcry was swift following The Post's report. Sen. Chris Van Hollen (D-Maryland) said the administration's potential use of AI to influence policy was dangerous.
'These people are unserious – but they pose a serious risk to Americans' health,' he wrote in a social media post.
Sen. Elizabeth Warren (D-Massachusetts) said in a statement, 'It's shameful that American parents even have to think about fake science and AI-generated studies in official White House reports on their kids' health.'
The entire episode is a 'cautionary tale' for the potential use of AI in government, said Anand Parekh, chief medical adviser at the Bipartisan Policy Center, a Washington, D.C., think tank.
'Did they not have enough staff?' he asked Friday. 'What are the checks?'
One reference in the initial version of the report cited a study titled 'Overprescribing of Oral Corticosteroids for Children With Asthma' to buttress the idea that children are overmedicated. But that study didn't appear to exist. There is a similar Pediatrics article from 2017 with the same first author but different co-authors. Later Thursday, that Pediatrics article was swapped in for the apparently nonexistent study in the version of the report available online.
An article credited to U.S. News & World Report about children's recess and exercise time was initially cited twice to support claims of declining physical activity among U.S. children, once with only part of the link shown. It listed Mlynek, A. and Spiegel, S. as different authors. Neither referred to Kate Rix, who wrote the story. Neither Mlynek nor Spiegel appear to be actual reporters for the publication. As of Thursday evening, Rix had been swapped in as the author on one of the references in the version of the report available online.
Nearly half of the 522 citations in the initial version of the report included links to articles or studies. But a Post analysis of all the report's references found that at least 21 of those links were dead.
Former governor and current New York City mayoral front-runner Andrew M. Cuomo was caught up in controversy last month after a housing policy report he issued used ChatGPT and garbled a reference. Attorneys have faced sanctions for using nonexistent case citations created by ChatGPT in legal briefs.
The garbled scientific citations betray subpar science and undermine the credibility of the report, said Georges C. Benjamin, executive director of the American Public Health Association.
'This is not an evidence-based report, and for all practical purposes, it should be junked at this point,' he said. 'It cannot be used for any policymaking. It cannot even be used for any serious discussion, because you can't believe what's in it.'
When asked about the nonexistent citations at a news briefing Thursday, press secretary Karoline Leavitt said the White House has 'complete confidence in Secretary Kennedy and his team at HHS.'
'I understand there were some formatting issues with the MAHA Report that are being addressed, and the report will be updated, but it does not negate the substance of the report, which, as you know, is one of the most transformative health reports that has ever been released by the federal government, and is backed on good science that has never been recognized by the federal government,' Leavitt said.
At some point between 1 p.m. and 2:30 p.m. Thursday, the MAHA Report file was updated on the White House site to remove mentions of 'corrected hyperlinks' and one of the 'oaicite' markers. Another 'oaicite' marker, attached to a New York Times Wirecutter story about baby formula, was still present in the document until it was removed Thursday evening. The White House continued to update the report into the night.
Department of Health and Human Services spokesman Andrew Nixon said that 'minor citation and formatting errors have been corrected, but the substance of the MAHA report remains the same – a historic and transformative assessment by the federal government to understand the chronic disease epidemic afflicting our nation's children.'
'Under President Trump and Secretary Kennedy, our federal government is no longer ignoring this crisis, and it's time for the media to also focus on what matters,' Nixon said.
Kennedy has long vowed to use AI to make America's health care better and more efficient, recently stating in a congressional hearing that he had even seen an AI nurse prototype 'that could revolutionize health delivery in rural areas.'
Peter Lurie, president of the nonprofit Center for Science in the Public Interest, said he was not surprised by the presence of possible AI markers in the report. Lurie said he had asked his own staff to look into it after noticing that the report linked to one of his organization's fact sheets but credited the Department of Agriculture and HHS as the authors.
'The idea that they would envelop themselves in the shroud of scientific excellence while producing a report that relies heavily on AI is just shockingly hypocritical,' said Lurie, who was a top Food and Drug Administration official in the Obama administration, where he wrote such government reports.
There are many pitfalls in modern AI, which is 'happy to make up citations,' said Steven Piantadosi, a professor in psychology and neuroscience at the University of California at Berkeley.
'The problem with current AI is that it's not trustworthy, so it's just based on statistical associations and dependencies,' he said. 'It has no notion of ground truth, no notion of … a rigorous logical or statistical argument. It has no notions of evidence and how strongly to weigh one kind of evidence versus another. '
The Post previously reported that the document stretched the boundaries of science with some of its conclusions. Several sections offer misleading representations of findings in scientific papers.