The Prototype: Chatbots Struggle With Analogies

Forbes

04-04-2025

In this week's edition of The Prototype, we look at the ways chatbots struggle to think like humans, how nanoparticles might connect brains and computers, a better way to vaccinate for food poisoning and more. You can sign up to get The Prototype in your inbox here.
One thing that humans are very good at is the ability to reason by analogy–the ability to look at a bunch of different situations or sets of facts and generalize from them. This kind of thinking is vital in a number of fields–especially science and the law.
But as it turns out, large language models like ChatGPT aren't very good at this at all. That's according to a new paper published in Transactions on Machine Learning Research, which found that humans significantly outperformed GPT-3 and GPT-4 on analogical reasoning on different kinds of problems.
The paper also suggests that even when such models appear to 'reason,' that performance is very sensitive to whether it's been trained on similar problems, whereas humans are able to make good generalizations even about subjects they're not familiar with.
This is a similar outcome to another paper that came out in January, which found that while large language models can diagnose disease very well when presented with medical exam-style questions, they're not very good at all when it comes to diagnosing diseases from patient conversations.
As many of the AI companies are starting to move beyond just large language models and into applications that utilize multiple types of AI, it'll be interesting to see whether their performance at abstraction is able to improve, or if they'll continue to be limited by the robustness of their training data.
Stay tuned.
Tetiana Aleksandrova, founder & CEO of Subsense
Subsense
For people who are paralyzed or have neurodegenerative diseases, brain-computer interfaces offer a way to regain some measure of their independence and improve their communication.
But these interfaces also offer challenges. Some, like Elon Musk's Neuralink, require a chip to be implanted directly into the brain. And while non-invasive approaches are being developed, open questions remain about how refined they can be due to the difficulty in reading brain signals through the skull.
Tetiana Aleksandrova thinks that her company, Subsense, may have an attractive compromise. It's developing a way to use nanoparticles made from gold and polymers that are injected into and bind to particular areas of the brain, making it possible to better monitor brain activity without surgery. The technology can also stimulate the brain, which has potential to treat a number of neurological diseases as well as improve coordination with prosthetic limbs.
The company, which recently emerged from stealth with $17 million in seed funding, has already tested its technology on mice, demonstrating the ability to read brain signals and stimulate neural activity. The company plans to spend the next couple of years developing and scaling its technology.
'Our end goal is helping individuals to enhance their capabilities,' Aleksandrova told me. 'To basically hack your brain, but in a good way.'
Developing vaccines against disease-causing bacteria in the gut, like salmonella, has long been challenging because the bacteria living there varies so much from person to person. But researchers at ETH Zurich may have a solution–they gave mice a vaccine against salmonella as well as a probiotic containing benign bacteria strains that compete for its resources. The scientists found that this combination helped fight disease both by triggering the immune system to attack salmonella and enabling the good bacteria to outcompete it. Next up is seeing if this approach works in people. Their findings were published this week in Science.
On Monday, SpaceX launched its sixth private mission flying people into orbit, called Fram2. Funded by Chun Wang, a cryptocurrency entrepreneur, the four people on board are the first people to ever fly in an orbit over both the North and South Poles. The crew is also performing 22 different scientific experiments on behalf of institutions from eight different countries, including studying aurora in the atmosphere, growing mushrooms in microgravity and more. The spacecraft is planned to splash down off the California coast later today.
Aetherflux, founded by former Robinhood cofounder Baiju Bhatt, announced that it has raised $50 million in a series A round led by Index Ventures and Interlagos to support its mission to develop satellite-based solar power. The company also stated that it's been approved for Department of Energy funding to develop a proof of concept for power transmission from orbit. I first spoke to Bhatt last year to discuss his solar power plans in-depth, and you can read that story here.
I talked to billionaire Sean Parker about his plans for scaling up his non-profit institute devoted to cancer research, which has spent over $300 million on nearly 500 academic projects that have resulted in over 4,000 papers, 300 patents, dozens of clinical trials and 17 new biotech startups to capitalize on this research.
In my other newsletter, InnovationRx, my colleague Amy Feldman and I looked at the Trump administration's massive health agency layoffs, Palantir's lawsuit against a YC-backed startup, Gather Health's new primary care model and more.
SpaceX's valuation hit $350 billion in December, which Forbes estimates makes its president and COO Gwynne Shotwell a billionaire.
Researchers at Washington State University developed a machine learning model that can help predict where new pandemics might emerge from interactions between people and animals.
A new blood test for Alzheimer's disease helps doctors both diagnose the condition in patients and determine how far the dementia it causes has progressed.
The International Space Station National Laboratory has launched a new startup accelerator program to aid entrepreneurs in developing new space businesses.
Biotechnology company Oxitec has broken ground on a new facility in Brazil, which will breed mosquitos infected with Wolbachia bacteria, which reduces their ability to transmit diseases like dengue and Zika.
Lithium-ion batteries are great for electronics but it's well-known that over time their capacity slowly degrades. This was thought to be mostly due to overcharging them, but as it turns out, the degradation actually accelerates if a battery is used for too long without recharging. So if you want to extend the battery life in your phone, don't let the level get too low before you plug it in again. The research findings were published in Advanced Energy Materials.
I'm currently reading The Rise and Fall of the Galactic Empire by historian Chris Kempshall. It's presented as an in-universe text detailing the history of the titular empire over the course of the Star Wars movies and TV shows. It's an engrossing account that draws upon how Star Wars reflects the ways democracies can fall into authoritarianism in real history, and how that history can repeat itself. One particularly chilling quote in it: 'This is an important lesson for the galaxy moving forward: there will always be those who are prepared to accelerate the death of democracy if they believe power is being given to someone worthy.'
If you're a Star Wars fan, it's well worth the read.