454 Hints That a Chatbot Wrote Part of a Biomedical Researcher's Paper

New York Times

02-07-2025

Scientists know it is happening, even if they don't do it themselves. Some of their peers are using chatbots, like ChatGPT, to write all or part of their papers.
In a paper published Wednesday in the journal Science Advances, Dmitry Kobak of the University of Tübingen and his colleagues report that they found a way to track how often researchers are using artificial intelligence chatbots to write the abstracts of their papers. The A.I. tools, they say, tend to use certain words — like 'delves,' 'crucial,' 'potential,' 'significant' and 'important' — far more often than human authors do.
The group analyzed word use in the abstracts of more than 15 million biomedical abstracts published between 2010 and 2024, enabling them to spot the rising frequency of certain words in abstracts.
The findings tap into a debate in the sciences over when it is and is not appropriate to use A.I. helpers for writing papers.
When ChatGPT was introduced in November 2022, a collection of words started showing up with unusual frequency. Those words, the investigators report, were not used so often before the release of ChatGPT. They infer that the change in word usage is a telltale sign of A.I.
In 2024, there were a total of 454 words used excessively by chatbots, the researchers report. Based on the frequency of the A.I.-favored words, Dr. Kobak and his team calculate that at least 13.5 percent of all biomedical abstracts appeared to have been written with the help of chatbots. And as many as 40 percent of abstracts by authors from some countries writing in a few less selective journals were A.I.-generated.
Want all of The Times? Subscribe.