Behind the Curtain: Your smarter fake friends

Axios

13 hours ago

Your fake friends are getting a lot smarter ... and realer.
Why it matters: If you think those make-believe people on Facebook, Instagram and X — the bots — seem real and worrisome now, just wait.
Soon, thanks to AI, those fake friends will analyze your feeds, emotions, and habits so they can interact with the same savvy as the realest of people.
The next generation of bots will build psychological profiles on you — and potentially billions of others — and like, comment and interact the same as normal people.
This'll demand even more vigilance in determining what — and who — is real in the digital world.
A taste of the future: Brett Goldstein and Brett Benson — professors at Vanderbilt University who specialize in national and international security — show in vivid detail, in a recent New York Times op-ed, the looming danger of the increasingly savvy fake world.
They dug through piles of documents uncovered by Vanderbilt's Institute of National Security, exposing how a Chinese company — GoLaxy — optimizes fake people to dupe and deceive.
"What sets GoLaxy apart," the professors write, "is its integration of generative A.I. with enormous troves of personal data. Its systems continually mine social media platforms to build dynamic psychological profiles. Its content is customized to a person's values, beliefs, emotional tendencies and vulnerabilities."
They add that according to the documents, AI personas "can then engage users in what appears to be a conversation — content that feels authentic, adapts in real-time and avoids detection. The result is a highly efficient propaganda engine that's designed to be nearly indistinguishable from legitimate online interaction, delivered instantaneously at a scale never before achieved."
Between the lines: This makes Russia's bot farms look like the horse and buggy of online manipulation. We're talking real-time adaptations to match your moods, or desires, or beliefs — the very things that make most of us easy prey.
The threat of smarter, more realistic fake friends transcends malicious actors trying to warp your sense of politics — or reality. It hits your most personal inner thoughts and struggles.
State of play: AI is getting better, faster at mimicking human nuance, empathy and connection.
Some states, including Utah and Illinois, are racing to limit AI therapy. But most aren't. So all of our fake friends are about to grow lots more plentiful.
A Harvard Business Review study ($) earlier this year found the number one use case of chat-based generative AI is therapy ("structured support and guidance to process psychological challenges") and companionship ("social and emotional connection, sometimes with a romantic dimension").
AI-based therapy, the article notes, is "available 24/7, it's relatively inexpensive (even free to use in some cases), and it comes without the prospect of judgment from another human being."
That research is congruent with what the biggest AI companies are finding: Humans are increasingly turning to AI to be buddies and shrinks. That brings a passel of possible problems — from unregulated robots offering bad advice, to unhealthy human attachment to an artificial thing.
The Wall Street Journal found by examining public chat transcripts that bots sometimes egg on users' false premises. To go along with AI hallucination, clinicians are informally calling this phenomenon " AI psychosis" or "AI delusion."
There's obvious upside, too: Loneliness can be deadly, and good therapy can do great things for someone struggling. Meta, as Axios reported in May, envisions chatbots as " more social" — potentially an extension of your friend network, and antidote to the " loneliness epidemic."
What you can do: Be vigilant. This is all happening now. It's safe to assume AI only gets better, and bad actors more clever. Don't assume every person online is real — much less a real friend.