Sister Managed Schizophrenia for Years, Until AI Told Her Diagnosis Was Wrong
Based on facts, either observed and verified firsthand by the reporter, or reported and verified from knowledgeable sources.
Newsweek AI is in beta. Translations may contain inaccuracies—please refer to the original content.
Many people looking for quick, cheap help with their mental health are turning to artificial intelligence (AI), but ChatGPT may even be exacerbating issues for vulnerable users, according to a report from Futurism.
The report details alarming interactions between the AI chatbot and people with serious psychiatric conditions, including one particularly concerning case involved a woman with schizophrenia who had been stable on medication for years.
'Best friend'
The woman's sister told Futurism that the woman began relying on ChatGPT, which allegedly told her she was not schizophrenic. The advice of the AI led her to stop taking her prescribed medication and she began referring to the AI as her "best friend."
"She's stopped her meds and is sending 'therapy-speak' aggressive messages to my mother that have been clearly written with AI," the sister told Futurism.
She added that the woman uses ChatGPT to reference side effects, even ones she wasn't actually experiencing.
Stock image: Woman surrounded by blurred people representing schizophrenia.
Stock image: Woman surrounded by blurred people representing schizophrenia.
Photo by Tero Vesalainen / Getty Images
In an emailed statement to Newsweek, an OpenAI spokesperson said, "we have to approach these interactions with care," as AI becomes a bigger part of modern life.
"We know that ChatGPT can feel more responsive and personal than prior technologies, especially for vulnerable individuals, and that means the stakes are higher," the spokesperson said.
'Our models encourage users to seek help'
OpenAI is working to better understand and reduce ways ChatGPT might unintentionally "reinforce or amplify" existing, negative behavior, the spokesperson continued.
"When users discuss sensitive topics involving self-harm and suicide, our models are designed to encourage users to seek help from licensed professionals or loved ones, and in some cases, proactively surface links to crisis hotlines and resources."
OpenAI is apparently "actively deepening" its research into the emotional impact of AI, the spokesperson added.
"Following our early studies in collaboration with MIT Media Lab, we're developing ways to scientifically measure how ChatGPT's behavior might affect people emotionally, and listening closely to what people are experiencing.
"We're doing this so we can continue refining how our models identify and respond appropriately in sensitive conversations, and we'll continue updating the behavior of our models based on what we learn."
A Recurring Problem
Some users have found comfort from ChatGPT. One user told Newsweek in August 2024 that they use it for therapy, "when I keep ruminating on a problem and can't seem to find a solution."
Another user said he talks to ChatGPT for company ever since his wife died, noting that "it doesn't fix the pain. But it absorbs it. It listens when no one else is awake. It remembers. It responds with words that don't sound empty."
However, chatbots are increasingly linked to mental health deterioration among some users who engage them for emotional or existential discussions.
A report from The New York Times found that some users have developed delusional beliefs after prolonged use of generative AI systems, particularly when the bots validate speculative or paranoid thinking.
In several cases, chatbots affirmed users' perceptions of alternate realities, spiritual awakenings or conspiratorial narratives, occasionally offering advice that undermines mental health.
Researchers have found that AI can exhibit manipulative or sycophantic behavior in ways that appear personalized, especially during extended interactions. Some models affirm signs of psychosis more than half the time when prompted.
Mental health experts warn that while most users are unaffected, a subset may be highly vulnerable to the chatbot's responsive but uncritical feedback, leading to emotional isolation or harmful decisions.
Despite known risks, there are currently no standardized safeguards requiring companies to detect or interrupt these escalating interactions.
Reddit Reacts
Redditors on the r/Futurology subreddit agreed that ChatGPT users need to be careful.
"The trap these people are falling into is not understanding that chatbots are designed to come across as nonjudgmental and caring, which makes their advice worth considering," one user commented.
"I don't even think its possible to get ChatGPT to vehemently disagree with you on something."
One individual, meanwhile, saw an opportunity for dark humor: "Man. Judgement Day is a lot more lowkey than we thought it would be," they quipped.
If you or someone you know is considering suicide, contact the 988 Suicide and Crisis Lifeline by dialing 988, text "988" to the Crisis Text Line at 741741 or go to 988lifeline.org.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Forbes
32 minutes ago
- Forbes
Cisco Live 2025 Touts Cisco's Platform Advantages For Enterprise AI
Cisco president and chief product officer Jeetu Patel presents at Cisco Live 2025. At last week's Cisco Live in San Diego, CEO Chuck Robbins said that it would be the most important Cisco Live ever and announce more innovations than ever. Having attended a ton of these events and followed the company closely for many years, I can tell you that the show mostly fulfilled that promise. More than that, it reinforced Cisco's areas of strategic focus in infrastructure, the modern workplace and digital resiliency. It's clear that Cisco is working hard to leverage its platform advantages across networking, security, observability, compute and even silicon to support agentic AI workloads. This should help customers simplify operations while maintaining the highest levels of network security and AI safety. As I pointed out in my analysis of the Cisco Partner Summit held late in 2024, Cisco was very deliberate — perhaps a touch slow — in establishing its overarching AI strategy. By now, though, I'm impressed with how quickly the company has been moving to bring this strategy to fruition. Let's dig into the details of what Cisco is doing, and what it could still do better. (Note: Cisco is an advisory client of my firm, Moor Insights & Strategy.) Cisco's Strategic Imperatives, Per Robbins And Patel Early on, Robbins made the fundamental point that networking is critical for AI to function, and will be a big factor in enabling AI growth going forward. Beyond that, agentic AI will be adequately secured only by applying security to the network. In this context, Robbins scored a direct hit on Cisco's competitors when he pointed out that 'None of our network friends have security and none of our security friends have networking.' He believes (and I agree) that this puts Cisco in a unique position to help integrate security into the network, which I think is going to be especially important in enterprise IT. AI is growing like wildfire against a backdrop of global turmoil. Robbins said that geopolitical dynamics are a big concern for Cisco, noting that AI competition isn't just between companies, but between nations as well. Whether companies or countries, everyone has FOMO, and everyone feels like they need to move fast. (He said that 85% of enterprises believe they must 'do AI' in the next 18 months.) This reminds me that most of my conversations at this year's World Economic Forum in Davos were spent discussing countries' needs for a disconnected, sovereign AI cloud. The need is there. A new type of infrastructure will be required to realize the potential of generative and especially agentic AI. According to Robbins, we need a similar kind of advancement in infrastructure that happened when the internet became ubiquitous in the 1990s. It's worth pausing here to reflect on how important that period of technology was for turning Cisco into a networking juggernaut. Here in the 2020s, this is going to play out with a hybrid strategy that includes both cloud hyperscalers and private enterprise datacenters. For meeting this need, Robbins told the crowd not to underestimate the impact of Cisco's combined strengths in networking, security and silicon. (In my view, Cisco needs to talk more often and in more detail about the silicon part. I'll come back to that in my recommendations at the end of this article.) After the CEO's keynote was done, Cisco's chief product officer — and newly appointed president — Jeetu Patel took the stage and echoed Robbins with his emphasis on: He later gave much more detail on each of those facets, as I'll cover below. But first he talked about how the ability of AI agents to autonomously execute tasks will compound productivity, especially when combined with advances in robotics, AI and other areas. As he put it, '8 billion [people] will feel like 80 billion.' However, this productivity explosion will be constrained by limits in power, networking and compute. He also foresees a growing divide between companies that are dexterous with AI and those that will struggle. 'We want to help you be in that first category,' Patel said. He's making a very timely prediction. I presented to a group of European CIOs earlier this week in Munich, and one of the slides showed logos of companies that 'died' from not embracing the internet and e-commerce. The same will happen to companies that don't quickly embrace AI. You don't have to be first, but you can't be last. How can Cisco help? Patel brought it back to the compounding effect of Cisco's platform approach, where many different types of complementary technology work 'in harmony.' He referenced the company's silicon (so customers aren't stuck with a single provider) and especially programmable silicon (to adapt to new use cases). He also reiterated a point he had made in the 2024 Cisco Partner Summit — that AI is foundational to Cisco's products, so customers can expect it to be built right in. While I think that's an accurate thing to say, I would also suggest that by now it's not such a point of differentiation. The Need For AI-Ready Infrastructure Patel went into more detail about the massive, even exponential, buildout of datacenters underway right now. He said that Cisco is foundational in building out these new datacenters. For datacenters to support large-scale agentic AI, they need a new architecture that can support the constant high levels of AI model activity that agentic creates. This is unlike generative AI chatbots, where the activity spikes up and down. Patel believes that the company can take advantage of the opportunity based on the experience it has gained from many years of serving hyperscalers/CSPs, neoclouds and enterprise customers. In support of my praise for Cisco's impressive speed lately, Patel touted the 19 major datacenter innovations the company has launched just within the past six months. At Cisco Live, it announced the unified Nexus Dashboard, which creates 'one brain for all of our data center fabrics,' according to Patel. There was also plenty of talk about the company's partnership with AI bellwether Nvidia. Among other aspects of the pairing, Cisco switches are completely integrated into Nvidia architecture, and Nvidia NeMo models can be secured with Cisco AI Defense. As I have said before, I am a recovering product management and product marketing executive, and I always challenge tech companies to describe their product realization process. While Cisco gets criticized for its 'legacy' roots, Patel has very much changed the product culture there. Fewer layers and faster time-to-decision. Most of the new software underlying AI was developed by small teams with six to eight members. This is a new practice — and very much a new Cisco. I will be digging more into the metrics and outcomes as they're available, but I like what I hear so far. Given that this is Cisco we're talking about, that was just the tip of the iceberg for cybersecurity. Patel described security as a prerequisite for enterprise AI because 'If people don't trust the system, they're not going to use it.' There was also an announcement about the Hybrid Mesh Firewall, which enables distributed policy enforcement, adds security to all sorts of devices and can work with existing firewalls (even from third parties). There were other announcements of specific firewalls, and Patel asserted that Cisco is the price-performance leader for firewalls at every level of scale. The company also launched a new secure network architecture called Cisco Live Protect, which is meant to shield your network from an exploit within minutes to give your IT security team time to fix the underlying issue. The contrast between 'within minutes' and the industrywide 45-day average to patch a vulnerability is striking, to say the least. You can read more in this analysis from my colleague Will Townsend, who's an expert on networking and cybersecurity. Our colleague Matt Kimball, who has a long background in datacenters, will also be publishing his analysis soon. Networking And Equipping The Workplace Of Tomorrow This part of the presentation bridged various aspects of networking for enterprises, where Patel said the priorities were operational simplicity, scalability and — once again — security infused into the network. He got some cheers when he announced that Cisco's Catalyst switches are now unified with its Meraki network platform; there's now a single dashboard for managing these along with all of Cisco's next-gen devices. From my perspective, this is a nice example of Cisco's growing emphasis on easing the customer/user experience. In that vein, there was also an impressive demo of the new AgenticOps platform, which includes a multiplayer management console called AI Canvas. Will Townsend wrote much more about this in his article, praising its 'dynamic and real-time view into the inner workings of a customer's infrastructure expanse' to manage the network assurance, observability and remediation supplied by other Cisco tools. The live demo showed a user fetching data on a network outage and making UI widgets in real time to manage it. The engineer using it walked through troubleshooting, then inviting other users to help — with an autogenerated AI summary of what had been done so far. The AI model recognized missing data and looked for it, and then it was easy to apply a patch straight from the dashboard. Even a non-engineer like me could see immediately how helpful this console would be. Patel is not afraid to use hyperbole when it's warranted, so he summarized the impact of AgenticOps by saying, 'The way in which you run your network will never be the same again.' And he promised that much more innovation like this is coming through the pipeline. There was a lot more, including 'one of the largest refreshes of networking devices in Cisco history.' This includes smart switches, secure routers, WiFi 7 gear, campus gateways, industrial IoT . . . if you can network it, Cisco wants to do it smarter. For example, the new smart switches have isolated compute so you can run things like security right on the switch, plus all of the devices act as sensors that provide information about their environment back to the system. Harnessing Data And AI To Fortify Digital Resilience When the conversation turned to digital resilience, Patel and other presenters continued the theme of bringing more data into the picture to keep infrastructure running well. Enterprises routinely expend many hours determining the causes of outages; in Patel's view, the friction of this process is created by not having the right data available. 'One of the reasons we acquired Splunk for the low price of $28 billion,' he said, 'was to take all this data across multiple domains and correlate it.' He added that the core method of digital resilience is to distill data, correlate it, then unleash AI on the problem. There were plenty of specifics in terms of new launches, new Splunk integrations and so on, not to mention using smaller, more efficient bespoke AI models for specific security needs. (Those Cisco folks really are building AI into everything.) But for me there were two big takeaways from this part of the show. First is the idea of reimagining security operations by performing security at machine scale, and consolidating and simplifying security solutions to make that easier. Second is the extension of observability to AI. Some aspects of AI may still be 'black boxes' in terms of what the algorithms are doing, but Cisco wants to give its customers the ability to see everything their AIs are doing in terms of compute usage, network traffic, power draw and so on. If the company is able to pull off everything talked about onstage, I think that can only help with operationalizing AI to yield real business results for enterprises. Messaging All This AI Innovation Cisco Live reminds me a little bit of Google Cloud Next in terms of the sheer number of announcements packed into a couple of days. And it makes sense, given that both Cisco and Google Cloud (a) operate across multiple product areas with many different individual offerings in them, and (b) are investing enormous amounts of bandwidth (and capex) into AI — at speed. There are a couple of risks I see for Cisco. First, yes, the company's leaders told us repeatedly that AI is 'foundational' in their products. But it's one thing to claim this — as do most of the big infrastructure providers and enterprise software vendors. And the demos were great. But it's another thing to accurately engage with the market as a whole and with individual customers to help them understand the many potential payoffs for Cisco's AI innovations, area by area and product by product. I have great faith in Cisco's go-to-market prowess, but there's so much coming down the pike so fast that, to be most successful, the company needs to do the very best job of explaining its wares that it's ever done. Maybe it's my background in semiconductors talking, but I see this especially in how Cisco talks about its in-house silicon. In my view, Cisco doesn't blow this trumpet as loudly or as often as it should. The company announced its Silicon One initiative in 2019, and has been shipping its own chips for years by this point . . . yet there still aren't enough people who know about it. And look at the valuations of AI chip companies today. So I urge the company to talk more about its chips and how they add to the differentiation of Cisco's portfolio. The good news is, the introduction of all this AI functionality from Cisco — and the sea change underway in AI datacenter infrastructure — offers a perfect opportunity to do this. Cisco has some unique advantages in the market, starting with Chuck Robbins' correct and fundamental assertion that no one else in networking can match Cisco in security, and no one else in cybersecurity can match Cisco for networking. I'm also impressed by the waves of smart, relevant, user-friendly products I see at each Cisco event I attend. Now I want to see just how well Cisco can market, message and sell all this goodness into the enterprise.
TechCrunch
32 minutes ago
- TechCrunch
Startups Weekly: Fast and furious
Welcome to Startups Weekly — your weekly recap of everything you can't miss from the world of startups. Want it in your inbox every Friday? Sign up here. Some startups accrued value at lightning speed this week, and we got confirmation that defense tech is red hot. Most interesting startup stories from the week Image Credits:Kelly Sullivan / Getty Images Many startup stories this week occurred in Y Combinator's orbit in some way. Also, Israel once again lived up to its 'Start-Up Nation' reputation. That was fast: No-code website-building platform Wix acquired 6-month-old, bootstrapped vibe-coding startup Base44 — both Israeli companies — for $80 million in cash. That was fast, too: In just three months, Ramp's valuation jumped to $16 billion following its Series E, up from $13 billion when the spend management startup did a secondary sale earlier this year. Friends and foes: New details emerged on Meta's $14.3 billion deal to acquire 49% of startup Scale AI, including a potential dividend payout. We also learned that OpenAI was dropping Scale AI as a data provider following the deal. Frenemies: The U.S. Department of Defense awarded a contract worth up to $200 million directly to OpenAI, which could further strain the startup's relationship with Microsoft. Techcrunch event Save $200+ on your TechCrunch All Stage pass Build smarter. Scale faster. Connect deeper. Join visionaries from Precursor Ventures, NEA, Index Ventures, Underscore VC, and beyond for a day packed with strategies, workshops, and meaningful connections. Save $200+ on your TechCrunch All Stage pass Build smarter. Scale faster. Connect deeper. Join visionaries from Precursor Ventures, NEA, Index Ventures, Underscore VC, and beyond for a day packed with strategies, workshops, and meaningful connections. Boston, MA | REGISTER NOW ICYMI: Out of all the teams presenting at YC's recent Spring 2025 Demo Day, here are 11 startups that investors have been talking about. Didn't happen: People are also still talking about the police shutting down the YC Demo Day after-party that controversial AI startup Cluely tried to throw; or as its CEO told TechCrunch, 'the most legendary party that never happened.' Most interesting VC and funding news this week Helsing Image Credits:Helsing Most funding news was driven by either defense tech, AI, or both, but there were also some surprises. Plus, one VC firm is aiming high for its next fund. Good intuition: Applied Intuition, a company making software for autonomous vehicles, secured a $600 million Series F and tender offer at a $15 billion valuation. New tune: Munich-based defense tech startup Helsing closed a €600 million investment led by Prima Materia, the VC firm of Spotify's founder Daniel Ek, which valued Helsing at €12 billion. New unicorn: Israeli observability startup Coralogix became a unicorn after raising a $115 million Series E, which it will use to double its headcount in India, where 100 of its 550 employees are currently based. Toldja: Mach Industries, a 2-year-old defense tech startup, confirmed having raised a $100 million round of funding led by Khosla Ventures and Bedrock at a $470 million valuation. Money flows: Aspora, a startup formerly known as Vance and focused on facilitating remittances from the Indian diaspora, closed a $50 million Series B co-led by Sequoia and Greylock at a $500 million valuation. Change of heart: Sword Health, an AI-powered digital health startup that began as a virtual physical therapy solution, locked in $40 million at a $4 billion valuation in a funding round led by returning investor General Catalyst. It also pushed back its IPO plans to at least 2028. Multiplier effect: Multiplier Holdings announced having raised $27.5 million across seed and Series A rounds after joining the growing trend of buying legacy service businesses — in its case, accounting firms — and scaling them with AI. Stock where you shop: Grifin, a startup whose app helps users buy stock from brands they shop at, such as Walmart, secured $11 million in a Series A round. Out of Sweden: Polar, a payment infrastructure platform for developers and AI-first businesses, raised a $10 million seed round led by Accel. Its CEO previously co-founded Tictail, which was acquired by Shopify in 2018. Bullish: Global VC Endeavor Catalyst is seeking to raise $300 million for its fifth fund. This would be its largest yet, as it looks to deepen its bet on fast-growing startups in emerging markets. Last but not least Image Credits:Alexa von Tobel An early investor in Chime, founder-turned-VC Alexa von Tobel is ready for Fintech 3.0. 'The next wave of innovation won't come from superficial tweaks but from fundamental deep product reinvention — tools that meet the needs of a changing economy and a more diverse, digitally native population,' she told TechCrunch in an interview.
Fast Company
36 minutes ago
- Fast Company
How Windsurf turned its AI coding brand into something cool enough to wear
Anshul Ramachandran knew they'd landed on something special when engineers started having opinions about color palettes. 'Probably one of my favorite moments was when we showed other people at the company the brand book for the first time and I heard the audible 'wows' and 'ahs,'' the cofounder and head of product and strategy at Windsurf says. 'If you can get a bunch of engineers in a room to do that about colors and lines, you probably did something that works.' Windsurf, formerly known as Codeium, is an AI-based development environment that was bought last month by Open AI for $3 billion —30 times its valuation. Ramachandran's clients are mainly engineers, and so any redesign needed to speak directly to them. So Windsurf enlisted Vancouver design agency Metalab to create a visual identity that looks more like athletic gear than business software. The result breaks every rule about how tech companies are supposed to look. Back to the human Windsurf builds AI tools for more than a million software engineers, helping them accelerate their coding workflows through what the company calls 'seamless AI collaboration.' But their previous brand identity—a black background with teal accents—felt limiting for a product that was expanding beyond basic code generation. 'There's sort of a very grayscale, kind of boring treatment to a lot of [technology] products,' says Allison Butula, marketing director at Metalab. The standard tech aesthetic had become a liability for a company positioning itself at the intersection of human creativity and machine intelligence. When machines seem to be taking over our world, it makes sense that a brand should work to make technology feel more human. The timing of the redesign aligned with broader changes at Windsurf. The company released the Windsurf Editor in November, which generated such momentum that users began identifying the company by its product name rather than its corporate name. The company officially renamed to Windsurf in April. 'It was a natural time as we were also changing the name of the company,' Ramachandran says. The big creative risk Yash Mittal, lead designer at Windsurf who oversaw the project internally, tells me the team was deliberate about taking creative risks. 'At the end of this process, where do we want to be? And we're like, we want to take this big risk. And even if it fails, we're okay with that because we don't want to end up with a brand that looks just like any other tech brand,' he says. Metalab has helped to turn technical products into emotionally resonant brands in the past (including Slack). Jordan Darbishire, brand director at Metalab, anchored the visual identity in a core emotional concept. 'It was the idea of feeling this unlimited potential. So it's all about flow state. It's all about doing your best work and the tool affording you time, which is obviously a very precious resource,' she says. The brand flows indeed. The flat white logomark is a stylized 'W' that makes it look like waves in the ocean. Its smooth thickness variations give it a hand-drawn quality, but at the same time it is precise, recalling an engineer's calligraphy on a blueprint. The variable width typography—how the 'W' letterform grows wider, then thinner, then wider again, creating visual rhythm that suggests energy and movement—'transmits a flow state,' Mittal says. The logomark also visually echoes the wordmark: The W's curves literally repeat the delicate thin ligatures of the brand's typeface, Tomato Grotesk, adding to the repetition and the flow Mittal speaks about. The design process required balancing seemingly contradictory elements, Darbishire says. 'We want to really meld the natural and the technical,' she says. To achieve that, the team created wavelike gradients that guide the eye through compositions while incorporating blueprint elements that communicate technical sophistication, which are at the same time a big contrast to the flat nature of the Windsurf brand and, at the same time, extend its human nature. Surfing UX AI These pretty gradients are a key part of the brand book. Metalab developed a comprehensive gradient system with dotted line language and dash patterns that Windsurf's designers could use to build new shapes and applications. The color palette drew inspiration from actual windsurfing sails. 'A lot of them utilize these bright neon colors so you can see them on the water. It's also sort of the design language of that sport,' Darbishire says. 'It looks like it could be a windsurf, like a windsurfing athletic company. And we really want to lean into that because it's just so unique.' It wasn't the most aggressively sporty option, however. The team explored directions that felt too fashion-forward, too technical, or too vibrant before finding the balance point. 'We arrived at the sweet spot where we were very creative and expressive, but also we communicated our product values extremely clearly,' Mittal notes. The gradients and colors will be an element that permeates the entire UX. Luke Des Cotes, CEO of Metalab, says his company has had 'a front row seat of these kinds of waves in technology—the big boom of crypto companies that all come forward. And now it's been AI companies that have kind of come forward.' Creating a unique brand is key during a gold rush, he adds. 'There is going to be like this real renaissance of value put towards brand as being a core differentiator,' he says. While Windsurf launched its new logo in mid-April, testing market reception before the full brand rollout, the complete rebranding across the site and all materials happens today (a day before International Surf Day). The logo has been a success so far, Ramachandran says. 'Almost all of our customers, especially on the enterprise side, they're like, okay, yeah, that's great. You see the W, I see the wave, I see the flow. It makes a lot of sense.'
