Sci-fi fans voice real-world concerns about how AI is changing the information ecosystem

Geek Wire

3 days ago

BOT or NOT? This special series explores the evolving relationship between humans and machines, examining the ways that robots, artificial intelligence and automation are impacting our work and lives.
Panelists at Seattle Worldcon 2025 discuss the impact of artificial intelligence on societal structures. From left: Ramez Naam, computer scientist and author; graphic novelist Moriko Handford; Kaylea Champion, professor of computer science at the University of Washington at Bothell; fan-fiction writer Berlynn Wohl; and moderator Corey Frazier, a member of the Oregon Innovation Council. (GeekWire Photo / Alan Boyle)
Worries about the dark side of artificial intelligence are stirring up a lot of buzz at the world's premier science-fiction convention, taking place this week in Seattle, and not just because intelligent machines are standard plot devices in sci-fi tales.
When the organizers of Seattle Worldcon 2025 acknowledged in April that ChatGPT was used to vet potential panelists for the event, it caused such an outcry that the organizers ended up issuing an apology and redoing the process without AI tools. The episode also inspired some writers and fans to organize a one-day, AI-free alternative conference called ConCurrent Seattle, which takes place on Thursday near Worldcon's venue.
Frank Catalano, a former tech executive and GeekWire contributor who is participating in three Worldcon panels this week, said science-fiction writers and artists fear that generative AI tools are plagiarizing their works to create automated products. (Full disclosure: My own book, 'The Case for Pluto,' is among the thousands of copyrighted works that were used to train Meta's Llama 3 AI model.)
'For writers, AI is an existential issue,' Catalano said. 'I think it understandably freaks writers out, especially science-fiction writers, to think that there are people talking that their work is obsolete because of technology they once wrote about.'
The concerns go far beyond job security for science-fiction writers. The broader debate was front and center today at the first of several Worldcon panels focusing on the tech world's AI revolution.
'Historically, AI has gone through these hype periods, say, the '60s, the '80s, etc. So you have this sort of AI summer period, but they're followed by AI winter,' said Kaylea Champion, a computer science professor at the University of Washington at Bothell. 'We are in AI summer, which means winter is coming.'
Champion said that today's large language models, or LLMs, are only as good as the data that they're trained on. 'It's essentially, at its heart, a word guesser,' she said. One of the risks looming in the near future is a phenomenon known as model collapse, which can occur when the outside sources of training data run dry.
'AI gets worse when you feed it its own output,' Champion explained. 'So, you know, out comes the poo and you shovel it back in, and the next round is even worse. It's disgusting, right? But that's actually the truth.'
Berlynn Wohl, a science-fiction writer whose day job involves helping low-income people find legal help, said many people aren't yet able to distinguish between authentic information and 'AI slop.'
'If you've ever watched one of those videos of someone talking to their parents, and their parent thinks that every picture of a kitten dancing in a sombrero is somehow real, right? What we're moving toward is a world where most people do not understand what is even being shown to them,' Wohl said.
Some figures who are prominent on the political scene — for example, President Donald Trump and tech billionaire Elon Musk — go so far as to try tweaking large language models to further their own worldviews.
'For somebody who lives their whole life in China, preventing something like 'woke' AI is kind of the norm — like having social networks say, either through social engineering or through force, only what the state approves. That could be our future,' said Moriko Handford, an engineer and graphic-novel author.
Strangely enough, the chatbots backed by Musk (who pushed for the creation of an AI agent known as Grok) and Trump (who rolled out Truth Search AI) sometimes fail to follow the intent of their masters. 'Grok is really woke,' said Ramez Naam, a technologist and science-fiction writer.
Naam acknowledged that AI posed potential risks, but he said there were potential benefits as well. 'AI is going to lift people up in general, create more abundance,' he said. 'It's not a perfect thing, but it's going to create more wealth in a lot of ways.'
What's the next frontier for AI? Naam noted that Google's revenue stream from search engine ads is rapidly being cannibalized by its AI Overview feature. 'So, for monetization, it seems pretty inevitable that we're going to get product placement in our chat results,' he said.
Champion said the AI revolution is still at an early enough stage that it's possible to 'push our elected leaders in the directions we would like to see,' though the opportunity to do that may be closing rapidly.
Just last month, the White House issued an AI policy plan that seeks to head off regulations. Champion championed the opposing view, saying it will be essential for governments to take a greater role in regulating the industry.
'We need the government to tell corporations that they need to behave responsibly, and put guardrails on technologies [to ensure] that they're safe for human use,' she said. 'That includes emotional and psychological safety, and not just physical safety.'
So, who would be in charge of policing AI? Panel moderator Corey Frazier, an engineer and philanthropist who's a member of the Oregon Innovation Council, had a surprising answer.
'I would say the thing I'm probably most expecting or hoping for is leveraging AI to check the AI,' he said. 'Have an AI checker that checks the AI.'
Seattle Worldcon 2025 continues through Sunday, with most events taking place at the Seattle Convention Center's Summit building. Check Worldcon's membership page for information about attendance fees. ConCurrent Seattle offers a full day of events on Thursday at Union Arts Center. Admission is free, but registration is required.