29-04-2025
Coming up: Rights for "conscious" AI
The AI industry —convinced it's on the verge of developing AI that's self-aware — is beginning to talk about ways to protect the "welfare" of AI models, as if they were entities that deserve their own rights.
Why it matters: The assumption that today's generative AI tools are close to achieving consciousness is getting baked into the industry's thinking and planning — despite plenty of evidence that such an achievement is at best very far off.
Driving the news: Anthropic last week announced a new research program devoted to "model welfare."
"Now that models can communicate, relate, plan, problem-solve, and pursue goals — along with very many more characteristics we associate with people — we think it's time to address" whether we should be "concerned about the potential consciousness and experiences of the models themselves," the announcement said.
Between the lines: Anthropic raises the "possibility" that AI might become conscious, and adopts a stance of "humility."
In carefully hedging its position, Anthropic takes cues from the 2024 research paper that kicked off the "model welfare" debate and that's co-authored by Anthropic AI welfare researcher Kyle Fish.
The idea is to prepare companies, policy makers and the public to face ethical choices about how they treat AI tools if evidence emerges that the tools have become worthy of ethical treatment — "moral patients," in the paper's terminology.
The big picture: Researchers say they want the world to realize that this potential has moved out of the realm of science fiction and into the world of near-future scenarios.
Some commentators draw comparisons with the debate over animal welfare and rights, with the podcaster Dwarkesh Patel suggesting that "the digital equivalent of factory farming" could cause "suffering" among AIs.
Yes, but: For every researcher urging us to take "AI welfare" seriously, there's a skeptic who sees the subject as a new species of hype.
After the New York Times' Kevin Roose wrote a weekend column exploring these questions, Wall Street Journal tech columnist Christopher Mims wrote on Bluesky, "Stories like this are a form of uncritical advertising for AI companies."
"I understand the impulse to take them at their word when they wonder aloud if their giant matrix multiplication engines are about to become sentient, but it should be resisted," Mims added.
The intrigue: There's little doubt that advanced LLMs are capable of creating a verbal facade that resembles human consciousness.
But AI critics argue that this attractive storefront is missing most of the foundations of self-awareness — including any awareness that isn't purely reactive to user prompts.
AI lacks a body. It has no sense of time, no hunger, no need for rest or desire to reproduce.
It can't feel pain or joy or anything else. (It can output words that claim it is experiencing a feeling, but that's not the same thing.)
It's all frontal cortex and no limbic system — all electric impulse and no brain chemistry.
One can imagine science gradually assembling an AI consciousness by adding missing pieces — but not in the near-future time frame the AI industry is discussing.
Some experts' vision leaves little room even for that kind of breakthrough.
"LLMs are nothing more than models of the distribution of the word forms in their training data, with weights modified by post-training to produce somewhat different distribution," as AI critic Emily Bender recently put it on Bluesky.
Others are less certain, but worry that focusing on "AI welfare" now could be premature and divert us from more urgent questions about AI's potential harms.
The 2024 AI welfare paper raises a similar concern: "If we treated an even larger number of AI systems as welfare subjects and moral patients, then we could end up diverting essential resources away from vulnerable humans and other animals who really needed them, reducing our own ability to survive and flourish. And if these AI systems were in fact merely objects, then this sacrifice would be particularly pointless and tragic."
Flashback: Google's Blake Lemoine argued three years ago that an early LLM had achieved sentience — and eventually lost his job.
Our thought bubble: Whatever happens, we're all going to need to exercise our imaginations, and fiction is still the best scenario-exploration machine humanity has invented.
