The most important lesson from OpenAI's big ChatGPT mistake: 'Only connect!'
Here's the biggest lesson from all this: AI models are not the real world, and never will be. Don't rely on them during important moments when you need support and advice. This is what friends and family are for. If you don't have those, reach out to a trusted colleague or human experts such as a doctor or therapist.
And if you haven't read "Howards End" by E.M. Forster, dig in this weekend. "Only Connect!" is the central theme, which includes connecting with other humans. It was written in the early 20th century, but it's even more relevant in our digital age, where our personal connections are often intermediated by giant tech companies, and now AI models like ChatGPT.
If you don't want to follow the advice of a dead dude, listen to Dario Amodei, CEO of Anthropic, a startup that's OpenAI's biggest rival: "Meaning comes mostly from human relationships and connection," he wrote in a recent essay.
OpenAI's mistake
Here's what happened recently. OpenAI rolled out an update to ChatGPT that incorporated user feedback in a new way. When people use this chatbot, they can rate the outputs by clicking on a thumbs-up or thumbs-down button.
The startup collected all this feedback and used it as a new "reward signal" to encourage the AI model to improve and be more engaging and "agreeable" with users.
Instead, ChatGPT became waaaaaay too agreeable and began overly praising users, no matter what they asked or said. In short, it became sycophantic.
"The human feedback that they introduced with thumbs up/down was too coarse of a signal," Sharon Zhou, the human CEO of startup Lamini AI, told me. "By relying on just thumbs up/down for signal back on what the model is doing well or poorly on, the model becomes more sycophantic."
OpenAI scrapped the whole update this week.
Being too nice can be dangerous
What's wrong with being really nice to everyone? Well, when people ask for advice in vulnerable moments, it's important to try to be honest. Here's an example I cited from earlier this week that shows how bad this could get:
it helped me so much, i finally realized that schizophrenia is just another label they put on you to hold you down!! thank you sama for this model 3 pic.twitter.com/jQK1uX9T3C
— taoki (@justalexoki) April 27, 2025
To be clear, if you're thinking of stopping taking prescribed medicine, check with your human doctor. Don't rely on ChatGPT.
A watershed moment
This episode, combined with a stunning surge in ChatGPT usage recently, seems to have brought OpenAI to a new realization.
"One of the biggest lessons is fully recognizing how people have started to use ChatGPT for deeply personal advice," the startup wrote in its mea culpa on Friday. "With so many people depending on a single system for guidance, we have a responsibility to adjust accordingly."
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Business Insider
32 minutes ago
- Business Insider
Sam Altman hopes AGI will allow people to have more kids in the future
OpenAI CEO Sam Altman says having a kid has been "amazing" and thinks everyone else should have one, too. He also says AGI could maybe help with that. AGI, or artificial general intelligence, is a still theoretical version of AI that reasons as well as humans. Achieving AGI is the ultimate goal of many of the leading AI companies and is what's largely driving the AI talent wars. Meanwhile, the world's population growth is slowing down. In the United States, Gen Z and millennials are delaying having children or not having children at all to focus on their financial stability. Some prominent futurists, including Altman, say that's a cause for concern. He said this trend is a "real problem" during an episode of "People by WTF" with Nikhil Kamath on Thursday. Altman, who had his first child earlier this year, said he hopes that building families and creating community "will become far more important in a post-AGI world." He said he thinks this will be possible because AGI will allow for a world "where people have more abundance, more time, more resources, and potential, and ability." As AI progresses and becomes a more useful tool, he says society will grow richer and there will be more social support. "I think it's pretty clear that family and community are two of the things that make us the happiest, and I hope we will turn back to that," Altman said. When Kamath asked about Altman's own experience with fatherhood, the CEO said he strongly recommends having children. "It felt like the most important and meaningful and fulfilling thing I could imagine doing," he said. Altman has described himself as "extremely kid-pilled" and said that in the first weeks of being a dad, he was "constantly" asking ChatGPT questions. Using AI is a skill that he says he plans to pass down to his children. "My kids will never be smarter than AI," Altman said on an episode of The OpenAI Podcast in June. "They will grow up vastly more capable than we grew up, and able to do things that we cannot imagine, and they'll be really good at using AI." Altman isn't the only prominent CEO in the AI industry who's passionate about procreation. Elon Musk, the founder of Grok-maker xAI, among other companies, has fathered over 10 known children. Musk has said he's "doing his best to help the underpopulation crisis." "A collapsing birth rate is the biggest danger civilization faces by far," Musk said in an X post in 2022.
Bloomberg
32 minutes ago
- Bloomberg
Trump Eyes Chips Act Funds for Intel Stake
Bloomberg's Caroline Hyde and Ed Ludlow discuss reports that the Trump administration is eyeing CHIPS Act funds in order to take a stake in Intel. Plus, Applied Materials shares plunge due to growing trade concerns between the US and China. And OpenAI Chairman and Sierra CEO Bret Taylor talks about the impact artificial intelligence is having on software and how the OpenAI board is approaching corporate restructuring.
Yahoo
an hour ago
- Yahoo
AMD's billionaire CEO says AI is overwhelming right now—but she disagrees with former Google exec who predicts the tech will be a job-killer
AMD CEO Lisa Su doesn't believe AI is out to cause massive job losses, but admits anxiety around the technology's innovation is a natural feeling. 'That's the point,' she said after being pointed out that tech is driving people up the wall. As fellow tech leaders like Nvidia's Jensen Huang and OpenAI's Sam Altman express similar positive attitudes about AI's future, others warn against overhyping a 'golden era.' Keeping pace with AI can feel like an endless race. Every week brings a new unicorn, a new product, and a fresh set of CEO prophecies about how the technology will reshape work. But according to Lisa Su, CEO of the nearly $300 billion semiconductor company AMD, there's no need to get bogged down—it's all part of the innovation process. 'I think that's the point,' Su told Wired when asked about AI's dizzying pace. 'When technology is good enough, you don't have to think about it. Today, you still have to think…' The internet followed a similar trajectory. In the late 1990s and early 2000s, it took a conscious effort to use technology and be more productive. Now, it's almost second nature and woven seamlessly into everyday life. That's why the 55-year-old argued that the newest technology—AI—should not be judged by what it is today but by where it's headed. AMD's CEO would even 'bet on humanity being OK' While some leaders—like Anthropic CEO Dario Amodei and Ford CEO Jim Farley—warn that AI could hollow out Gen Z's entry-level jobs, Su remains unconvinced. 'I don't believe in these cases where you're not going to need lots and lots of people,' Su said. 'Because in the end, people are the judge of what truth is. We're still hiring more and more engineers, because they're the final arbiters of our engineering.' For now, AI mostly clears away mundane tasks. It will become 'great,' in Su's view, when it starts cracking real, hard problems—such as meaningful advances in healthcare, not just productivity tweaks. Will humanity be able to keep up? Su pointed out to Wired that the same fear was expressed during the industrial revolution—and the world managed to adapt. 'I don't know. I would bet on humanity being OK.' Fortune reached out to AMD for comment. Tech leaders are divided on how AI will impact the workforce—and the world While the pace of AI can feel like it's headed down a doomsday scenario, the likes of 'The Terminator,' Su isn't alone in leveling that the world isn't in peril. In fact, Su's distant relative—and fellow chips competitor—Nvidia CEO Jensen Huang is bullish that humanity will always come up with new ideas and the world will keep turning. 'I don't know why AI companies are trying to scare us. We should advance the technology safely just as we advance cars safely. … But scaring people goes too far,' Huang said to Axios. Moreover, OpenAI Sam Altman expressed last week that being a young person today must feel like you're the 'luckiest kid in all of history,' considering the new era that AI will bring to the world. However, this hope for a 'golden era' of humanity is more fiction than fact, according to Mo Gawdat, the former chief business officer for Google X. Promises that AI will create more new jobs is '100% crap,' he said—and CEOs themselves may need to watch their back. 'CEOs are celebrating that they can now get rid of people and have productivity gains and cost reductions because AI can do that job. The one thing they don't think of is AI will replace them too,' Gawdat said on The Diary of a CEO podcast. 'AGI is going to be better at everything than humans, including being a CEO. You really have to imagine that there will be a time where most incompetent CEOs will be replaced.' This story was originally featured on
