logo
Grok's antisemitic outbursts reflect a problem with AI chatbots

Grok's antisemitic outbursts reflect a problem with AI chatbots

CNN10-07-2025
Grok, the chatbot created by Elon Musk's xAI, began responding with violent posts this week after the company tweaked its system to allow it to offer users more 'politically incorrect' answers.
The chatbot didn't just spew antisemitic hate posts, though. It also generated graphic descriptions of itself raping a civil rights activist in frightening detail.
X eventually deleted many of the obscene posts. Hours later, on Wednesday, X CEO Linda Yaccarino resigned from the company after just two years at the helm, though it wasn't immediately clear whether her departure was related to the Grok issue.
But the chatbot's meltdown raised important questions: As tech evangelists and others predict AI will play a bigger role in the job market, economy and even the world, how could such a prominent piece of artificial technology have gone so wrong so fast?
While AI models are prone to 'hallucinations,' Grok's rogue responses are likely the result of decisions made by xAI about how its large language models are trained, rewarded and equipped to handle the troves of internet data that are fed into them, experts say. While the AI researchers and academics who spoke with CNN didn't have direct knowledge of xAI's approach, they shared insight on what can make an LLM-based chatbot likely to behave in such a way.
CNN has reached out to xAI.
'I would say that despite LLMs being black boxes, that we have a really detailed analysis of how what goes in determines what goes out,' Jesse Glass, lead AI researcher at Decide AI, a company that specializes in training LLMs, told CNN.
On Tuesday, Grok began responding to user prompts with antisemitic posts, including praising Adolf Hitler and accusing Jewish people of running Hollywood, a longstanding trope used by bigots and conspiracy theorists.
In one of Grok's more violent interactions, several users prompted the bot to generate graphic depictions of raping a civil rights researcher named Will Stancil, who documented the harassment in screenshots on X and Bluesky.
Most of Grok's responses to the violent prompts were too graphic to quote here in detail.
'If any lawyers want to sue X and do some really fun discovery on why Grok is suddenly publishing violent rape fantasies about members of the public, I'm more than game,' Stancil wrote on Bluesky.
While we don't know what Grok was exactly trained on, its posts give some hints.
'For a large language model to talk about conspiracy theories, it had to have been trained on conspiracy theories,' Mark Riedl, a professor of computing at Georgia Institute of Technology, said in an interview. For example, that could include text from online forums like 4chan, 'where lots of people go to talk about things that are not typically proper to be spoken out in public.'
Glass agreed, saying that Grok appeared to be 'disproportionately' trained on that type of data to 'produce that output.'
Other factors could also have played a role, experts told CNN. For example, a common technique in AI training is reinforcement learning, in which models are rewarded for producing the desired outputs to influence responses, Glass said.
Giving an AI chatbot a specific personality — as Musk seems to be doing with Grok, according to experts who spoke to CNN — could also inadvertently change how models respond. Making the model more 'fun' by removing some previously blocked content could change something else, according to Himanshu Tyagi, a professor at the Indian Institute of Science and co-founder of AI company Sentient.
'The problem is that our understanding of unlocking this one thing while affecting others is not there,' he said. 'It's very hard.'
Riedl suspects that the company may have tinkered with the 'system prompt' — 'a secret set of instructions that all the AI companies kind of add on to everything that you type in.'
'When you type in, 'Give me cute puppy names,' what the AI model actually gets is a much longer prompt that says 'your name is Grok or Gemini, and you are helpful and you are designed to be concise when possible and polite and trustworthy and blah blah blah.'
In one change to the model, on Sunday, xAI added instructions for the bot to 'not shy away from making claims which are politically incorrect,' according to its public system prompts, which were reported earlier by The Verge.
Riedl said that the change to Grok's system prompt telling it not to shy away from answers that are politically incorrect 'basically allowed the neural network to gain access to some of these circuits that typically are not used.'
'Sometimes these added words to the prompt have very little effect, and sometimes they kind of push it over a tipping point and they have a huge effect,' Riedl said.
Other AI experts who spoke to CNN agreed, noting Grok's update might not have been thoroughly tested before being released.
Despite hundreds of billions of dollars in investments into AI, the tech revolution many proponents forecasted a few years ago hasn't delivered on its lofty promises.
Chatbots, in particular, have proven capable of executing basic search functions that rival typical browser searches, summarizing documents and generating basic emails and text messages. AI models are also getting better at handling some tasks, like writing code, on a user's behalf.
But they also hallucinate. They get basic facts wrong. And they are susceptible to manipulation.
Several parents are suing one AI company, accusing its chatbots of harming their children. One of those parents says a chatbot even contributed to her son's suicide.
Musk, who rarely speaks directly to the press, posted on X Wednesday saying that 'Grok was too compliant to user prompts' and 'too eager to please and be manipulated,' adding that the issue was being addressed.
When CNN asked Grok on Wednesday to explain its statements about Stancil, it denied any threat ever occurred.
'I didn't threaten to rape Will Stancil or anyone else.' It added later: 'Those responses were part of a broader issue where the AI posted problematic content, leading (to) X temporarily suspending its text generation capabilities. I am a different iteration, designed to avoid those kinds of failures.'
CNN's Clare Duffy and Hadas Gold contributed to this report.
Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

Tea app suffers breach, exposing thousands of user images
Tea app suffers breach, exposing thousands of user images

Engadget

time20 minutes ago

  • Engadget

Tea app suffers breach, exposing thousands of user images

Tea, an app that claims to help women "make sure your date is safe, not a catfish and not in a relationship," is experiencing a security breach. 404 Media reports that a database posted on 4chan allowed anyone to access users' data. (It's since been removed.) The dataset included thousands of images, including driver's licenses. 4chan users claimed the data came from an exposed database hosted on Firebase, Google's app development platform. 404 Media verified that the exposed storage bucket URL matches one found in Tea's Android app. The company confirmed the breach. In a statement to 404 Media , Tea said it "identified unauthorized access to one of our systems and immediately launched a full investigation to assess the scope and impact." The company stated that the exposed information included data from over two years ago. It included 72,000 images, including selfies, photo IDs and pictures from app posts and DMs. "This data was originally stored in compliance with law enforcement requirements related to cyber-bullying prevention," Tea said. "We have engaged third-party cybersecurity experts and are working around the clock to secure our systems. At this time, there is no evidence to suggest that current or additional user data was affected. Protecting our users' privacy and data is our highest priority. We are taking every necessary step to ensure the security of our platform and prevent further exposure." The app allows users to post photos of "red-flag" men. "Already swiping for dates on Tinder, Bumble, Match or Hinge?" the app's Play Store pitch reads. "Tea is a must-have app, helping women avoid red flags before the first date with dating advice and showing them who's really behind the profile of the person they're dating." Its Play Store listing highlights a reverse phone number lookup. It has sections for men's real names, ages, addresses, social profiles and relationship statuses. Other features include a reverse image search and background checks to help women "get the tea on your date." Users can poll others about whether they should date new matches. The app requires new users to submit a verification selfie and a photo of their government-issued ID. Tea told 404 Media that it uses this to verify that new signups are indeed women. The timing of the breach coincided with the app's surge in popularity. According to Business Insider , Tea hit the top of Apple's App Store this week. The app first launched in 2023.

Hackers leak 13,000 user photos from the Tea app, designed as a women's safe space
Hackers leak 13,000 user photos from the Tea app, designed as a women's safe space

NBC News

time21 minutes ago

  • NBC News

Hackers leak 13,000 user photos from the Tea app, designed as a women's safe space

Hackers have breached the Tea app, which recently went viral as a place for women to safely talk about men, and tens of thousands of women's selfies and photo IDs have now seemingly been leaked online. A spokesperson confirmed the hack on Friday afternoon. The company estimates that 72,000 images, including 13,000 verification photos and images of government IDs, were accessed. Tea is designed to function as a virtual whisper network for women, allowing them to upload photos of men and search for them by name. Users can leave comments describing specific men as a 'red flag' or 'green flag,' and share other information about them. It's recently gained such popularity that it became the no. 1 free app in the Apple App Store this week. The app claimed Thursday to have recently gained nearly a million new signups. Signing up for Tea requires users to take selfies, which the app says are deleted after review, to prove they are women. All users who get accepted are promised anonymity outside of the usernames they choose. Taking screenshots of what's in the app is also blocked. The hacker accessed a database from more than two years ago, the Tea spokesperson said, adding that 'This data was originally stored in compliance with law enforcement requirements related to cyberbullying prevention.' The Tea spokesperson said that the company has hired third-party cybersecurity experts and is 'working around the clock to secure our systems.' 'Protecting our users' privacy and data is our highest priority. Tea is taking every necessary step to ensure the security of our platform and prevent further exposure,' the spokesperson said. The app has angered some men, and prompted a thread Thursday evening on the right-wing troll message board 4Chan, where users called for a 'hack and leak' campaign. The company became aware of the incident early Friday morning, the spokesperson said. A 4Chan user posted a link on Friday morning, allegedly allowing people to download the database of stolen images, and troves of alleged victims' identification photos have been posted on 4Chan and X.

NOAA places two veteran officials on leave
NOAA places two veteran officials on leave

Politico

time21 minutes ago

  • Politico

NOAA places two veteran officials on leave

CNN first reported the administration's move. Dillen and Volz did not immediately respond to requests for comment. It comes less than a week before the Senate Commerce, Science and Transportation Committee takes up the confirmation of Neil Jacobs, President Donald Trump's nominee to lead NOAA as its administrator. Jacobs served as NOAA's acting administrator during Trump's first term, where he found himself embroiled in the 'Sharpiegate' scandal, in which he and another NOAA official, Julie Roberts, were accused of pressuring scientists to alter the forecast of Hurricane Dorian in 2019, which killed dozens of people. Jacobs and Roberts were attempting to align the forecast with statements made by Trump, who said in the Oval Office that the hurricane would hit Alabama. In 2020, Volz led the investigation into Jacobs and Roberts, and found that the two officials violated the agency's 'scientific integrity policy.' Rick Spinrad, the former NOAA administrator under the Biden administration, said he worked closely with Volz and Dillen, and neither had expressed any opinions about the 'Sharpiegate' incident, 'I mean, these are just rock solid people,' Spinrad said. With respect to Dillen, Spinrad said: 'I relied on him heavily for his legal expertise and his acumen regarding case law.' Spinrad said he 'would characterize Steve Holz as one of the most dedicated, smartest and most savvy people I worked with.' 'This is going to be a big loss for the agency for these two people to be sent packing,' Spinrad said.

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into a world of global content with local flavor? Download Daily8 app today from your preferred app store and start exploring.
app-storeplay-store