logo
The Scientific Reason Why ChatGPT Leads You Down Rabbit Holes

The Scientific Reason Why ChatGPT Leads You Down Rabbit Holes

CNET2 days ago

That chatbot is only telling you what you want to believe, according to a new study.
Whether you're using a traditional search engine like Google or a conversational tool like OpenAI's ChatGPT, you tend to use terms that reflect your biases and perceptions, according to the study, published this spring in the Proceedings of the National Academy of Sciences. More importantly, search engines and chatbots often provide results that reinforce those beliefs, even if your intent is to learn more about the topic.
For example, imagine you're trying to learn about the health effects of drinking coffee every day. If you, like me, enjoy having exactly two cups of joe first thing in the morning, you may search for something like "is coffee healthy?" or "health benefits of coffee." If you're already skeptical (maybe a tea purist), you might search for "is coffee bad for you?" instead. The researchers found that framing of questions could skew the results -- I'd mostly get answers that show the benefits of coffee, while you'd get the opposite.
"When people look up information, whether it's Google or ChatGPT, they actually use search terms that reflect what they already believe," Eugina Leung, an assistant professor at Tulane University and lead author of the study, told me.
The abundance of AI chatbots, and the confident and customized results they so freely give you, makes it easier to fall down a rabbit hole and harder to realize you're in it. There's never been a more important time to think deeply about how you get information online.
The question is: How do you get the best answers?
Asking the wrong questions
The researchers conducted 21 studies with nearly 10,000 participants who were asked to conduct searches on certain preselected topics, including the health effects of caffeine, gas prices, crime rates, COVID-19 and nuclear energy. The search engines and tools used included Google, ChatGPT and custom-designed search engines and AI chatbots.
The researchers' results showed that what they called the "narrow search effect" was a function of both how people asked questions and how the tech platforms responded. People have a habit, in essence, of asking the wrong questions (or asking questions in the wrong way). They tended to use search terms or AI prompts that demonstrated what they already thought, and search engines and chatbots designed to provide narrow, extremely relevant answers, delivered on those answers. "The answers end up basically just confirming what they believe in the first place," Leung said.
Read more: AI Essentials: 29 Ways to Make Gen AI Work for You, According to Our Experts
The researchers also checked to see if participants changed their beliefs after conducting a search. When served a narrow selection of answers that largely confirmed their beliefs, they were unlikely to see significant changes. But when the researchers provided a custom-built search engine and chatbot designed to offer a broader array of answers, they were more likely to change.
Leung said platforms could provide users with the option of a broader, less tailored search, which could prove helpful in situations where the user is trying to find a wider variety of sources. "Our research is not trying to suggest that search engines or algorithms should always broaden their search results," she said. "I do think there is a lot of value in providing very focused and very narrow search results in certain situations."
3 ways to ask the right questions
If you want a broader array of answers to your questions, there are some things you can do, Leung said.
Be precise: Think specifically about what exactly it is you're trying to learn. Leung used an example of trying to decide if you want to invest in a particular company's stock. Asking if it's a good stock or a bad stock to buy will likely skew your results -- more positive news if you ask if it's good, more negative news if you ask if it's bad. Instead, try a single, more neutral search term. Or ask both terms and evaluate the results of each.
Get other views: Especially with an AI chatbot, you can ask for a broad range of perspectives directly in the prompt. If you want to know if you should keep drinking two cups of coffee a day, ask the chatbot for a variety of opinions and the evidence behind them. The researchers tried this in one of their experiments and found they got more variety in results. "We asked ChatGPT to provide different perspectives to answer the query from the participants and to provide as much evidence to back up those claims as possible," Leung said.
At some point, stop asking: Follow-up questions didn't work quite as well, Leung said. If those questions aren't getting broader answers, you may get the opposite effect -- even more narrow, affirming results. In many cases, people who asked lots of follow-up questions just "fell deeper down into the rabbit hole," she said.

Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

RFK Jr. taps allies and COVID vaccine critics among picks for CDC advisory panel. Here's who's on the list.
RFK Jr. taps allies and COVID vaccine critics among picks for CDC advisory panel. Here's who's on the list.

CBS News

time26 minutes ago

  • CBS News

RFK Jr. taps allies and COVID vaccine critics among picks for CDC advisory panel. Here's who's on the list.

Health and Human Services Robert F. Kennedy Jr. announced Wednesday he's naming eight new advisers to serve on the Centers for Disease Control and Prevention's vaccine recommendations committee, after firing the committee's entire previous roster of 17 advisers. "All of these individuals are committed to evidence-based medicine, gold-standard science, and common sense. They have each committed to demanding definitive safety and efficacy data before making any new vaccine recommendations," Kennedy said Wednesday in a post on X. Kennedy's picks circumvented the usual CDC process for selecting members of the Advisory Committee on Immunization Practices. In previous administrations, career agency officials — not political leaders — vetted potential experts before forwarding them to the department for the secretary's approval. The panel's influential recommendations are closely watched because they are directly tied to federal policies, like which vaccines insurers are required to cover. The picks announced by Kennedy include some close allies of the secretary and his inner circle. One of them, Dr. Robert Malone, worked on early research related to mRNA vaccine technology but was accused during the COVID-19 pandemic of spreading misinformation about the mRNA vaccines. He was with Kennedy and President Trump at the Trump election night celebration in Florida. "On the basis of data from all over the world, approximately three years ago it was my impression that the risk/benefit ratio of these products did not merit continued use in any cohort," Malone posted last month on his Substack about the mRNA COVID vaccines. Like Kennedy, Malone has questioned the benefits of measles vaccines during the recent record outbreak in Texas, which killed two children, and he has promoted unproven treatments for the virus. Another member picked by Kennedy is Dr. Martin Kulldorff, an epidemiologist who co-authored the pandemic-era Great Barrington Declaration criticizing COVID-19 restrictions, along with now-NIH Director Dr. Jay Bhattacharya. Bhattacharya has described Kulldorff as a close friend. Kulldorff previously worked with the CDC's outside vaccine advisers, before authoring an opinion piece in 2021 criticizing the agency's decision to pause use of Johnson & Johnson's COVID-19 vaccine over safety concerns. He claimed he was fired from working with the committee over the opinion piece. Kulldorff later claimed he was fired from Harvard University for criticizing COVID-19 vaccine requirements. Dr. Cody Meissner, a pediatrics professor who previously served as a member of the Food and Drug Administration's own vaccines panel — the Vaccines and Related Biological Products Advisory Committee – was also named to the committee. Meissner opposed COVID-19 vaccine requirements for children. He also co-authored an opinion piece with now-FDA Commissioner Dr. Marty Makary speaking out against masking of children during the pandemic. Another former member of the FDA vaccines panel who was picked by Kennedy is Vicky Pebsworth, a regional director of the National Association of Catholic Nurses. Pebsworth spoke at a 2020 meeting of the FDA vaccines committee, where she identified herself as the research director for the National Vaccine Information Center and "mother of a child injured by his 15-month well-baby shots in 1998." She said the center's position was that any "coercion and sanctions to persuade adults to take an experimental vaccine, or give it to their children, is unethical and unlawful." Kennedy also praised another pick, MIT professor Retsef Levi, saying: "Dr. Levi has collaborated with public health agencies to evaluate vaccine safety, including co-authoring studies on mRNA COVID-19 vaccines and their association with cardiovascular risks." Levi previously called for more detailed data from the COVID-19 vaccine trials, suggesting that changes to how Pfizer's shot was produced may have caused side effects. But Levi faced criticism for a paper co-authored with Florida Surgeon General Dr. Joseph Ladapo, which was cited in the state's move to recommend that young men not get mRNA COVID-19 vaccines. Experts condemned the paper for misleading methods that could inflate the risk. The views on vaccines of several of Kennedy's other picks are less clear. Kennedy said Dr. Michael A. Ross "contributed to national strategies for cancer prevention and early detection, including those involving HPV immunization," working with the CDC's breast and cervical cancer committee. Ross is described by Kennedy as an obstetrics and gynecology professor at George Washington University and Virginia Commonwealth University, though his name does not appear on directories for either university. Spokespeople for the two institutions did not immediately respond to a request for comment. Dr. James Pagano, described by Kennedy as a "strong advocate for evidence-based medicine," appears to have published little about vaccines or medicine. Records from the Medical Board of California list Pagano as being retired. Another Kennedy pick, Dr. Joseph Hibbeln, retired from the National Institutes of Health in 2020. His research portfolio previously covered nutritional intake of fatty acids like omega-3. Kennedy described him as bringing "expertise in immune-related outcomes, psychiatric conditions, and evidence-based public health strategies."

Tiny fragment of asteroid giving Field Museum scientists a glimpse 4.6 billion years into the past
Tiny fragment of asteroid giving Field Museum scientists a glimpse 4.6 billion years into the past

CBS News

timean hour ago

  • CBS News

Tiny fragment of asteroid giving Field Museum scientists a glimpse 4.6 billion years into the past

The Field Museum is the new temporary home to a tiny piece of pristine asteroid. The fragment of the asteroid Bennu, on loan from NASA, won't be on display for visitors, but will give scientists the chance to study an asteroid sample uncontaminated by Earth's atmosphere. A tiny, black fragment might not seem exciting, until a scientist explains it's a specimen from space. "It's an honor of a lifetime to be able to study this sample," said Field Museum curator Dr. Philipp Heck. How did Heck feel when the little rock first arrived at the museum and he held the vial containing the sample? "It was amazing. I was looking forward to that moment for a long time," he said. NASA's OSIRIS-REx mission was planned decades ago. In 2016, a spacecraft launched. In 2018, it arrived at Bennu, a near-Earth asteroid as wide as the Sears Tower is tall. The mission collected pieces of the asteroid and brought them back to Earth in 2023. "This is the first U.S. mission that sends a spacecraft to the asteroid and brings a sample back to Earth," said University of Chicago graduate student Yuke Zheng, who is part of the OSIRIS-REx sample analysis team. "It's a tiny, dark, black fragment that is fragile, so we want to protect it very carefully." She'll use the museum's scanning electron microscope to get an up-close look at a tiny sample of Bennu. "What struck me is how dark the sample is. I had never seen such a dark sample," Heck said. The fragment is like a time capsule, taking scientists back 4.6 billion years. "We believe Bennu contains part of the ingredients for life, and part of the ingredients of the formation of Earth," Heck said. Suddenly, a fragment at the bottom of a vial can have you pondering your place in the universe. "I've never studied a pristine sample from an asteroid," Heck said.

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into the world of global news and events? Download our app today from your preferred app store and start exploring.
app-storeplay-store