The new reputation risk: When AI misquotes you

Fast Company

18-07-2025

AI summaries are everywhere now. They appear above search engine results, in chatbots, and as part of the tools customers use when deciding which brands to trust. But what happens when those summaries are wrong, and they're the first thing people see about you?
Welcome to the new era of hallucinated reputation.
AI OVERVIEWS ARE FAST—AND FLAWED
In May 2024, Google launched AI Overviews to millions of U.S. users. The tool uses generative AI to answer search queries with a synthesized answer pulled from across the web. But within days, the cracks showed.
One user asked how to make cheese stick to pizza. Google's AI Overview replied, 'Add a little non-toxic glue.' That response was pulled from a sarcastic Reddit post, not a food blog or scientific source. AI Overviews has treated satirical websites as if they were genuine news outlets and given detailed answers to nonsense questions like 'Can you lick a badger twice?'.
With AI-powered search and summaries being integrated into other major platforms, these issues aren't simple one-offs. Far from rare, their impact will have lasting consequences.
BAD DATA IN, FALSE NARRATIVES OUT
Unlike traditional search results that link directly to verified content, most AI systems don't cite reliable sources consistently. In some cases, they cite them incorrectly; other times, they just make things up. That's a big problem for businesses and public figures.
At erase.com, for example, we worked with a local HVAC service provider that shared a similar name with a much larger national provider, so AI Overviews and chatbot responses were blending the two. When someone searched 'Is [Company Name] legit?' or 'reviews for [Company Name],' the summaries pulled in Reddit threads and Quora comments about the national chain, even after adding narrower search terms like the local business' city or suburb. The generated reviews cited complaints and service issues that had nothing to do with the local provider. One summary even listed an out-of-state headquarters address and misidentified the owners.
AI hallucinations aren't the only concern. Fake reputation services are now bundling AI tools into their offerings. Need a dozen fake press mentions? They'll generate them with GPT-based models and upload them to abandoned blogs. Need a fake bio or testimonial? It's just a prompt away. These fabricated mention often pass initial detection and are hard to trace.
This isn't just about bad jokes or trivia. It's about trust. When customers, investors, or media search for your name or company and see an AI summary that's flat-out wrong, it can quickly distort public perception. And because these AI summaries are designed to reduce clicks, users may never scroll past the top answer to see a correction.
In 2024, the Federal Trade Commission (FTC) issued a final ruling to ban the creation and sale of fake reviews and deceptive testimonials, including those generated by AI. Meanwhile, the U.K.'s Competition and Markets Authority pressured Google into tougher content moderation policies, including warning labels for businesses that manipulate reviews. The EU's Digital Services Act (DSA) now mandates transparency around algorithmic recommendation systems.
But these enforcement systems are still playing catch-up. Hallucinated AI summaries don't clearly fall into categories like defamation or review fraud. They're just wrong. And right now, no one is really accountable. The burden shouldn't just fall on the occasional business that gets caught. There needs to be more accountability for the platforms that enable this kind of manipulation in the first place. Government fines help, but they're only hitting a fraction of the problem.
WHAT YOU CAN DO ABOUT IT
Reputation isn't just about what people say anymore. It's about what machines think people say. Your online presence is being read, summarized, and judged by large language models. And your reputation is being reduced to a few AI-generated lines. If those lines are wrong, misleading, or based on outdated content, your brand or name can take a hit before you even know what's out there. This is the new reality, and you can't opt out. But you can influence what AI sees.
The first step is to audit your public presence. Search your name or brand across major search platforms and AI tools with browsing enabled. Take note of summaries, bios, or claims that are inaccurate or outdated. If something's wrong, don't just wait for it to go away—address it.
Then, update your bios, About pages, and press mentions with consistent facts. Make sure the first three to five search results for your name reflect who you are today, not who you were years ago. These are the signals AI pulls from most often.
It's not just about ranking anymore. People are asking chatbots and AI tools things like 'Is [Your Name] legit?' or 'What do people say about [Your Company]?' That's why long-tail keywords and conversational Q&A content matter. AI models prioritize natural language and structured answers when summarizing reputations.
In a world where machines write the first draft of your reputation, the real risk is staying silent. AI doesn't understand nuance. It just summarizes whatever is most visible and frequent. Make sure the version of you they find is the one you actually want seen.