24-04-2025
A Discovery Every Day: What Does Superintelligence Actually Look Like?
As we start heading toward a certain critical mass with artificial intelligence, this word keeps coming up – superintelligence.
It's easy to throw the word around, and talk about that point when AI becomes smarter than humans, but what does superintelligence actually look like?
To find out, I asked a panel of builders and physicists, and we talked about what types of efforts are happening now to support superintelligent results. These are some of the main themes.
One of the biggest ideas in today's tech world is the idea of reinventing the scientific process itself.
Geoffrey von Maltzahn is part of the team at Flagship Pioneering, where a project called Lila Sciences is looking to create 'autonomous science systems' that will, as he says, allow AI to take over 'every step of the wheel' when it comes to scientific discovery.
'(It's) the ability to call upon tools, to model the way that the world works, to propose a brilliant hypothesis, to autonomously design a decisive experiment, (and) to test that hypothesis in the real world,' he clarified.
Geoffrey von Maltzahn, CEO, Lila
John Werner
Making the analogy from vibe coding to outsourcing, von Maltzahn talked about how even though human level intelligence supports everything, it's possible to bring together autonomous results for aspects like material science, chemistry, and life sciences.
That, he suggested, will really have a positive effect, partly because of inherent human limitations in scientific inquiry.
'Neither our bodies, nor our brains, are really optimally suited for science,' he said, 'particularly learning about how the world works from the atoms make the same case for math. You know, try as our brains have … in the reality in life science, in chemistry, materials and more, our brains really struggle to understand what is going on … but machines are much better at matching those patterns … the implications for every single technology domain that we're familiar with are really, really amazing.'
Panel on Superintelligence
John Werner
Here's another major part of what superintelligence is likely to be able to do: it will excel at math, even in more intuitive, abstract ways.
Carina Hong, CEO of Axiom, a quantitative superintelligence moonshot company, talked about how pattern matching is not reasoning, and how traditional models don't excel in showing their work.
'Large language models, despite all the amazing post training breakthroughs, are still pretty bad at doing proofs,' she said. 'They will give you a numerical answer. In fact, they can do it really well on the American Invitational math examination. Frontier large language models achieve a 96% score. However, when you ask (the model) to show its proof, the score drops to 5%, so why is different? It's because of the way we train them … what we want to build at Axiom is to use programming language to train the machine to be able to speak the language of formal proof.'
This, she says, will enable humans to trust the result of these engines, and make the world 'math-rich'.
Another aspect of this is the setup.
Riccardo Sabbatini is a numerical modeling specialist who works on drug discovery and more.
Setting the stage for full robotic automation, he talked about a system where millions of molecular experiments can happen with no human involvement whatsoever.
'I see a transition moment between now and super intelligence,' he said, calling the interim a time of 'vibe intelligence.'
'When you look at a coder today,' he said, 'instead of going and searching in Stack Overflow every three seconds, and having to copy and paste from (one's) own old code, you have open on the right side of your screen, an LLM: this is going to do boiler plating for you. It's going to like 80% of the boring coding that has been done in the past.'
You can watch the video for some additional scientific assessment of things like probabilistic database design, Gaussian curves, and the evolution of AI math.
One anecdote from the panel is where Sabbatini talked about image creation models always displaying watches with the same time setting – 10 minutes after 10 o'clock.
It's persistent, he said, based on the training set that the LLM gets off of the Internet.
'None of (the generated watches) will show 2:25pm,' he said of an experiment where a user asks for an image of a watch set at this time in the afternoon. 'They will always show 10:10; the reason is that the majority, if not the complete, set, of pictures of watches in the entire world, points at 10:10.'
It's an advertising thing, he suggested, based on how people like to see watch faces.
'So any watch in the world has to be at 10:10, stuck there,' he said. 'You can have a pretty analog watch. You can have a classic analog watch - but you can't have a '2:25' analog watch. It is bizarre, if you think about it, such a simple concept learning thing.'
That illustrates some of AI's current blind spots that the panel suggested might be solved with superintelligence, eventually.
But what von Maltzahn said about the pace of scientific discovery was extremely interesting.
With these new tools, he reasoned, we'll be able to speed up science as a human in endeavor: where a ground truth used to take about a year to develop, AI will free us of those time constraints.
What if you could have a breakthrough scientific discovery every day? And do it easily?
'The human brain understands a really, really small fraction of how the world works,' von Maltzahn explained. 'And in fact, to understand it, we've been dividing it into sub, sub, sub specialties. So I believe something like imagination and taste for novelty is going to hang around as a human contribution for a while.'
'I believe science is going to get way more fun,' von Maltzahn said. 'If you just take the Edisonian 1% inspiration, 99% perspiration, you know, we can put 99% perspiration into a new paradigm … (improving) the quality of life for scientists, and likely the quantity of output.'
Panel on Superintelligence
John Werner
Talking about being able to source rare earth metals in new ways, and perfect a new system of chemistry that's going to change our supply chains and our scientific methods, he suggested that the 'GDP of civilization' is resting on a brand new paradigm.
We've had any number of technological revolutions in the past, he argued, but this is the first intelligence revolution. What's going to happen?
One such outcome, posited by von Maltzahn as he discussed changes, is that none of our human intellectual contributions to projects will be safe, if AI can do it better.
'None of us really knows in what order the sea level of intelligence will rise and subsume imagination or … logical derivation,' he said. 'But there's probably a rough boundary where, if searching for information within the repository of what is known, then that is underwater now, or will be underwater virtually immediately.'
That brings me back to the eternal specter of job displacement, and the question of how we're going to re-order society around these technologies.
We seem to have a vague idea that a re-ordering is needed, but not much clarity on what people are going to be doing for jobs in a business world that's dominated by capable AI.
In any case, we can anticipate the likelihood of this new era of science, and everything that is going to bring us. This is something every young person should be thinking about as they study and prepare for a career – and something every public planner (or innovator, or entrepreneur) should be thinking about as they try to understand where we're going next.
