7 hours ago
Study: ChatGPT's creativity gap
AI can generate a larger volume of creative ideas than any human, but those ideas are too much alike, according to research newly published in Nature Human Behavior.
Why it matters: AI makers say their tools are "great for brainstorming," but experts find that chatbots produce a more limited range of ideas than a group of humans.
How it works: Study participants were asked to brainstorm product ideas for a toy involving a brick and a fan, using either ChatGPT, their own ideas, or their ideas combined with web searches.
Ninety-four percent of ideas from those who used ChatGPT "shared overlapping concepts."
Participants who used their own ideas with the help of web searches produced the most "unique concepts," meaning a group of one or more ideas that did not overlap with any other ideas in the set.
Researchers used ChatGPT 3.5 and ChatGPT-4 and reported that while ChatGPT-4 is creating more diverse ideas than 3.5, it still falls short ("by a lot") relative to humans.
Case in point: Nine participants using ChatGPT independently named their toy "Build-a-Breeze Castle."
The big picture: Wharton professors Gideon Nave and Christian Terwiesch and Wharton researcher Lennart Meincke found that subjects came up with a broader range of creative ideas when they used their own thoughts and web searches, compared to when they used ChatGPT.
Groups that used ChatGPT tended to converge on similar concepts, reducing overall idea diversity.
"We're not talking about diversity as a DEI type of diversity," Terwiesch told Axios. "We're talking about diversity in terms of the ideas being different from each in biology, we need a diverse ecosystem."
Zoom in: A 2024 study found similar results.
Participants were asked to write short fiction with and without ChatGPT.
Generative AI–enabled stories were found to be more similar to each other than stories by humans.
Yes, but: ChatGPT can be used as part of the brainstorming process. Terwiesch says idea variance comes from using ChatGPT to generate ideas, while also coming up with your own ideas and collecting original ideas from others.
Terwiesch also recommends "chain of thought prompting," which means asking your chatbot to generate several ideas, but also specifically asking the bot to make those ideas different from each other.
"If I just sit back and let ChatGPT do the work, I'm not taking the full advantage of what this tool has to offer. I can do better than that," Terwiesch told Axios.
A spokesperson from OpenAI shared best practices for prompting ChatGPT, advice from writers on how to use the tool and a student's guide to writing with ChatGPT.
