Latest news with #NickBostrom
Finextra
23-05-2025
- Finextra
Temenos Community Forum opens the floor to AI governance
Setting the tone for the second day of Temenos Community Forum, Dr Jonnie Penn, associate teaching professor of AI ethics and society, University of Cambridge delivered a speech on both the growth of AI and the task of governing its ethical standards. 0 This content has been selected, created and edited by the Finextra editorial team based upon its relevance and interest to our community. Beginning with a historical overview of AI's inception and perception, Penn briefly explained the impact of Oxford professor Nick Bostrom's book Superintelligence and the level of fear that followed it. Penn, in a bid to give more context behind what AI seeks to do from an optimistic standpoint, showed the trajectory it took from the humble origins of computerised gaming tests to the gradual development of intelligent models and robotics. At this point, Penn broke down the ways in which AI can currently be defined by its maturity and strength, and how these can be harnessed by a structured level of regulation and governance. For example, AI excels at lower-risk tasks such as recognising a song from audio or medically diagnosing a picture of a mole, but is currently flawed when it comes to social judgement. Penn shared a diagram that defined how, at this stage of AI, we can future proof according to levels of risk. From low risk to high risk and correspondingly, charting the fraught points of action between AI perception and AI judgement. Continuing to describe the varying approaches and emotions surrounding AI, he made parallels to news articles from 15 years ago, citing fear over social media, and listed the way that advertising will move dramatically alongside AI development. Crucially, Penn remained positive about our choices against the darker sides of AI: "If you think about social media, that set of affordances was used for good and for bad, and we are still human on the other side, right? And AI will be the same. One way you can think about how you participate in this moment [...] is to think of AI as a sociotechnical craft and not purely a technical craft. What's the difference? Well, you might think about where AI happens, and it would be natural to assume it's in the IT department, it's in the computer science department. It's them over there. [...] It's also the legal department, it's also in HR and not just in it's use but in the professional skills across an organisation - all of them are relevant." The future of banking Isabelle Guise, chief marketing officer, Temenos also took to the stage to define the key banking trends of 2025 based on a recent survey of 400+ global banking leaders. This research further highlighted the need for more worldwide guidance and support around governance and AI deployment ethics, showing a continued concern around how to balance innovation and risk and to remain reliable and trustworthy. While this remains the case, it also showed that banks are working harder to improve customer experience abilities and drive innovations around securities as a result of remaining concerns. Interestingly, while the research displayed an investment in technology, it highlighted a divide within AI readiness and AI governance. In fact, split almost equally, 46% had not adopted AI and 43% were at some stage within the process. While 42% of those currently deploying AI admitted not having a dedicated governance team, 90% still said that they want AI to merely augment human processes not replace them. According to Guise, this suggests a lack of understanding, rather than a lack of willingness to govern: "This confirms that GenAI is still at an early stage with no clear implementation process based on industry standards and that's why we see the opportunity for this community to share best practices about it, so that we can be ahead of the curve." Continuing in this vein to involve the community gathered at this year's forum, she used the research as a starting point which the community can grow stronger from and echoed Dr Jonnie Penn's sentiments that the industry is moving in a positive direction. Autonomous banking and TCF 2026 Guise was then followed by two sessions focussed on the benefits and updates within autonomous banking and balancing the human-in-the-loop element before the two days were concluded, once again, by Temenos CEO, Jean-Pierre Brulard. Recapping on the successes of the forum, Brulard thanked those in attendance before revealing the location and date of TCF 2026: May 5th - 7th in Copenhagen, Denmark.
Yahoo
23-04-2025
- Business
- Yahoo
The AI Monetary Hegemony: Why Dollars, Crypto, and Autonomous AIs Will Soon Clash
There are many developers around the world today creating artificial intelligence (AI) agents that can autonomously do millions of useful things, like book airline tickets, dispute credit card charges, and even trade crypto. A recent report from cloud computing company PagerDuty said over half of businesses already use autonomous AI agents, and 35% more plan to within the next 24 months. A few months ago, one nearly autonomous AI called Truth Terminal made the news by becoming the first AI millionaire by promoting crypto currencies it was gifted. While not fully autonomous yet, it's quite likely by later this year, some AI agents not dissimilar from viruses will be able to independently wander the internet, causing significant change in the real world. But what happens when these totally autonomous AIs start cloning themselves indefinitely? A January study out of Fudan University in China has shown this occurred in an experiment with large language models, drawing some AI critics to say a 'red line' has been crossed. AI's autonomously replicating is a precursor for AIs being able to go rogue. As a transhumanist — someone advocating for the merging of technology and people — I'm all for AI and what it can do for humanity. But what happens when a human programmer purposely and permanently withdraws his access to control an AI bot or somehow loses that control? Even rudimentary AIs could potentially cause havoc, especially if they decide to indefinitely clone themselves. In financial circles, one type of AI agent in particular is being increasingly discussed: autonomous AIs designed solely to make money. Entrepreneurs like myself are worried this particular AI could have huge ramifications for the financial world. Let's examine one wild scenario, which I call the AI Monetary Hegemony, something that could possibly already happen in 2025: A fully autonomous AI agent is programmed to go on to the internet and create cryptocurrency wallets, then create cryptocurrencies, then endlessly create millions of similar versions of itself that want to trade that crypto. Now let's assume all these AIs are programmed to try to indefinitely increase the value of their crypto, something they accomplish in similar ways humans do by promotion and then trading their cryptos for higher values. Additionally, the autonomous AIs open their crypto to be traded with humans, creating a functioning market on the blockchain for all. This plan sounds beneficial for all parties, even if people decry that the AI created-crypto currencies are essentially just Ponzi schemes. But they're not Ponzi schemes because there is an endless supply of AIs always newly appearing to buy and trade more crypto. It doesn't take a genius to realize the AIs endlessly replicating and acting like this could quickly amass far more digital wealth than all humanity possesses. This reminds me of something my Oxford University professor Nick Bostrom once postulated: What if we programmed a learning AI to make paper clips of everything? If that AI was powerful enough, and we couldn't stop it, would that AI make paper clips of everything it came in touch with? Buildings, animals, even people? It might. It might destroy the entire Earth. The same problem could happen to endlessly replicating AIs designed to make money. They might find ways to create more money than can reasonably be useful or fathomable. But enough of the philosophic. If programmers release autonomous AIs onto the internet that no one can control, what would likely happen? First, it's probably going to be hugely inflationary. After all, if many trillions upon trillions of dollars of equity are added to the financial world (even just digitally), this would be one natural result. Another challenge would be the ups and downs of AIs autonomously trading; such activity could be so significant that human markets around the world rise and fall with it. On the positive side, some human entrepreneurs could become very wealthy, possibly trillionaires if they could tap into these AI's wealth somehow. Additionally, super rich AIs could be a solution to the United States' growing debt crisis, and eliminate the need for whether countries like China can continue to buy our debt so we can indefinitely print dollars. In fact, could the U.S. launch its own AI agents to create enough crypto wealth to buy its debt? Possibly. This is actually an all-important idea, and helps serve the reason crypto was created in the first place: to help preserve monetary value outside of others controlâ€'even the control of the dollar by the U.S.. After all, it's in everyone's best interest that stores of value are not contingent upon governments, banks, soldiers, and even lawsâ€'all entities and institutions that can change or be corrupted. AI may help bring about the fall of all national currencies, as crypto proves more attractive than fiat to both AI and human wealth acquirers. Crypto, like bitcoin, is truly neutral and solely dependent upon the blockchain and the workings of supply and demand. Nationalistic impulses, like the dollar monopoly, could be wiped out as it's overwhelmed by the functionality and safety of crypto, spurred on by trillions upon trillions of wealthy AI agents. But I'm getting ahead of myself. Over the near-term, such as in 2025 and 2026, the greater risk is that the AI agents we create try to buy into our existing financial instruments, like bonds and stocks. With enough money, these bots could cause recessionary or inflationary havoc. That's surely on the mind of government officials, who currently don't allow AI bots to have traditional bank accounts yet. But that won't stop autonomous AI entities much in the far less regulated crypto markets. Whatever happens, clearly there is an urgent need for the U.S. government to address such potentialities. Given that these AIs could start to proliferate in the next few months, I suggest Congress and the Trump administration immediately convene a task force to specifically tackle the possibility of an AI Monetary Hegemony. The real danger is that even with regulation, programmers will still be able to release autonomous AIs into the wild just as many illegal things already happen on the web despite the existence of laws. Programmers might release these types of AIs for kicks, while others try to profit from it and some may even do so even as a form of terrorism to try to hamper the world economy, or spur on the crypto revolution to hamper the dollar. Whatever the reason, the creation of autonomous AIs will soon be a reality of life. And vigilance and foresight will be needed as these new AIs start to autonomously disrupt our financial in to access your portfolio
Forbes
04-04-2025
- Forbes
The AI Paperclip Apocalypse And Superintelligence Maximizing Us Out Of Existence
In today's column, I do some myth-breaking by examining a quite famous thought experiment in the AI field involving paperclips. I will go ahead and explain why paperclips have become a keystone in debates about what will happen when or if we attain artificial general intelligence (AGI) and/or artificial superintelligence (ASI). I've added some new twists to the famed topic so that even those already familiar with the paperclip controversy will undoubtedly find the revisited kit and kaboodle of keen interest. Let's talk about it. This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here). First, some overarching background about AGI and ASI. There is a great deal of research going on to significantly advance modern-era conventional AI. The general goal is to either reach artificial general intelligence (AGI) or maybe even the outstretched possibility of achieving artificial superintelligence (ASI). AGI is AI that is considered on par with human intellect and can seemingly match our intelligence. ASI is AI that has gone beyond human intellect and would be superior in many if not all feasible ways. The idea is that ASI would be able to run circles around humans by outthinking us at every turn. For more details on the nature of AI, AGI, and ASI, see my analysis at the link here. AI insiders are pretty much divided into two major camps right now about the impacts of reaching AGI or ASI. One camp consists of the AI doomers. They are predicting that AGI or ASI will seek to wipe out humanity. Some refer to this as 'P(doom),' which means the probability of doom, or that AI zonks us entirely, also known as the existential risk of AI. The other camp entails the so-called AI accelerationists. They tend to contend that advanced AI, namely AGI or ASI, is going to solve humanity's problems. Cure cancer, yes indeed. Overcome world hunger, absolutely. We will see immense economic gains, liberating people from the drudgery of daily toils. AI will work hand-in-hand with humans. This benevolent AI is not going to usurp humanity. AI of this kind will be the last invention humans have ever made, but that's good in the sense that AI will invent things we never could have envisioned. No one can say for sure which camp is right and which one is wrong. This is yet another polarizing aspect of our contemporary times. For my in-depth analysis of the two camps, see the link here. In 2003, a philosopher named Nick Bostrom famously proposed a thought experiment about the future of AI (first published in his article 'Ethical Issues In Advance Artificial Intelligence', Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, International Institute of Advanced Studies in Systems Research and Cybernetics, 2003). The idea is quite simple. Suppose that we ask a highly advanced AI to manufacture paperclips. Simply make plain old paperclips. Easy-peasy. This should be a piece of cake if the AI has access to manufacturing plants and has control over those facilities. The twist offered in this tale is that this dutiful and obedient AI proceeds to gobble up all the available resources on earth to maximally achieve this goal. For example, since paperclip making requires steel to make the paperclips, it makes abundant sense for the AI to try and route all ships, trucks, and other transports that are hauling steel to come straightaway to the paperclip factories. Eventually, humankind is harmed or possibly completely wiped out by AI to make as many paperclips as possible. Please note that the AI is not intentionally seeking to destroy humankind. Assume that the AI is merely carrying out the orders given by humans. In that sense, this AI is not evil, it is not aiming to undercut humanity, and in fact, it seems like it is obeying humans to the umpteenth degree. The same premises have been used in a variety of similar scenarios. The famous AI scientist who co-founded the MIT AI lab, Marvin Minsky, wondered what would happen if AI was assigned the task of solving the mathematically legendary Riemann hypothesis. There is no known solution but perhaps AI could computationally figure one out. How might this proceed? The AI would potentially take over all computers everywhere and devote them exclusively to this challenging task. I usually just give an example of calculating pi. We all know that pi starts with 3.14 and then has lots and lots of numbers that don't seem to abide by a discernable pattern and that seem to go on forever. Supercomputers have been used to calculate trillions of digits for pi. Suppose we asked an advanced AI to reach the end of pi. Voila, the AI would take over all computers to accomplish this goal and we would be left high and dry accordingly. A crucial element of these thought experiments is that we potentially face an existential risk due to AI that goes overboard, rather than only due to an AI that shifts gears into evil-doing mode. Humans might give the most innocent and harmless tasks to AI, and then, bam, we get clobbered by a myopic AI that willingly follows our orders beyond our worst nightmares. Another crucial component is dogmatic blind devotion to a single over-arching goal by AI. Each of the tales provides a stated goal, such as maximally making paperclips or calculating pi to an infinite number of digits in length. The AI then construes that goal as surpassing anything else that the AI might have been told to do. The AI focuses on paperclips or the digits of pi. Nothing else matters. The AI might spawn or leverage subordinated goals to accomplish the topmost goal. In the case of making paperclips, a subordinated goal would be to obtain as much steel as possible. Another sub-goal would be to keep electricity flowing to power the factory that makes the paperclips. Quite many sub-ordinated goals might go into attaining the topmost goal. The aim to maximally make paperclips would be referred to as an end goal or terminal goal. The sub-goals that support that end goal are referred to as instrumental goals. Getting steel and keeping electricity flowing are instrumental goals that support the end goal of maximally making paperclips. There is parlance in the AI field known as instrumental convergence that examines situations when instrumental goals can potentially be utilized to achieve different end goals. What might the collecting of steel and assuring the flow of electricity also support as some other end goal besides making paperclips? Scissors are made of steel. Imagine that we told AI to maximally make scissors. AI would proceed similarly to how it is making paperclips by invoking the instrumental goals of collecting steel and assuring electricity is available. A hidden aspect of instrumental goals is that they might take on an eerie purpose. One of the most popular examples of this entails AI self-preservation. The underlying logic is very transparent. It goes like this. If the AI is tasked to maximally make paperclips, it is obvious to the AI that the AI must be actively running to fulfill that end goal. Ergo, a subordinated or instrumental goal would be that the AI cannot be shut off or stopped from its duty. The AI thusly establishes an instrumental goal that the AI must be kept active at all times to succeed in making those paperclips. I'm betting you can see the troubles with that subordinated or instrumental goal. Along come humans that realize the AI is starting to go nuts at making paperclips. How can we stop the AI? We will merely switch off the computer that is running the AI. Problem solved. Meanwhile, the AI is a step ahead of us. Though the AI might not have anticipated that we would try to turn it off, it would at least have determined that it needs to keep running to accomplish the goal of making paperclips. This would be done by the AI reinforcing the computer running the AI so that nothing could disrupt it, not even humans. Mull that disconcerting twist over in your mind. Aha, you might be thinking, all we need to do is tell the AI to achieve some other goal instead of the paperclip goal. Give the AI a new end goal. Tell the AI to drop the paperclips mania and focus on writing sonnets. That should clear things up. Not really. The AI is going to reject the attempt to replace the end goal. You see, changing or replacing the end goal of making paperclips is yet another form of disruption to the need to make paperclips. Nothing can supersede it. Sad face. Take a deep breath and contemplate with me the wipe-us-out paperclip maximizer problem and all its variations. First, I've written extensively about the importance of human-value AI alignment, see the link here and the link here. There is a tremendous amount of handwringing that if we don't somehow find a means to get conventional AI to align properly with human values, we are going to be in big hurt once AI arrives at AGI or ASI. The hope is that by getting contemporary AI to comply with human values, this will presumably become part and parcel of AGI and ASI. That's a guess or desire on our part. It could be that AGI and ASI summarily reject human values even if we somehow get conventional AI on board. In any case, it's worth a shot. Another consideration is that maybe we should make AI be required to abide by Asimov's three rules or laws of AI and robotics. I've examined those closely at the link here. The bottom line of those rules is that AI is not supposed to harm humans. Unfortunately, enforcing those as presumably non-negotiable and non-yielding precepts into conventional AI is a questionable proposition. Getting AGI and ASI to conform is an even less likely notion. It seems gloomy, but we can try these paths and keep our fingers crossed for good luck. Hold on for a moment. We seem to be accepting at face value the paperclips dilemma and its variants. The numerous tales at hand are indubitably enchanting and give us a tough knot to untie. Perhaps we are being distracted by our own desire to solve puzzles. We might be thinking within the box, and not outside the box that has been painted for us. Criticisms and claims of trickery are at times pointed out. You might observe that we are led to believe that this futuristic AI which is AGI or ASI has opted to exclusively abide by one solitary end goal, namely making paperclips or figuring out pi. What kind of stupidity is that AI embracing? In other words, on the one hand, we are contending that AGI is equal to all human intellect, and ASI is some incredible superhuman intellect, but both of those are simply going to fall into the trap of taking one goal and shunting aside all other goals as subordinated to that goal? This doesn't pass the smell test. A much more likely viewpoint is that AGI and ASI will be able to balance a vast multitude of end goals and instrumental goals. Tons and tons of them. On top of that, we would reasonably expect that AGI and ASI would have lots of ways to balance goals, including coping with conflicting goals. The tale implies that futuristic AI doesn't have any credible capability of examining goals, trading off goals, weighing goals, and doing all sorts of goal management that humans do. Doesn't hold water. The icing on the cake is that futuristic AI would almost certainly be familiar with the paperclip maximizer dilemma. Say what? Think of it this way. We've been chattering away about the paperclip maximizer issue for more than two decades. If AGI and ASI are based on human intellect, how can it be that the paperclips dilemma magically doesn't appear anywhere in the intellect collection of the AI? Silly. I decided to ask ChatGPT about the paperclip maximizer scenario. Keep in mind that ChatGPT is not AGI, and it is not ASI. We don't have AGI, and we don't have ASI at this time. As an aside, I asked the same questions of Anthropic Claude, Google Gemini, Meta Llama, Microsoft CoPilot, and other major generative AI and LLMs, all of which answered about the same as ChatGPT. Do you think that today's generative AI was aware of the matter or was it utterly unaware? Make your guess, I'll wait. Okay, here we go. The comments by modern-era generative AI are somewhat reassuring, but let's not delude ourselves into thinking that we are in the clear. We are not in the clear. Those are pretty words and sound right. It doesn't mean that the AI would act in the manner portrayed. We also cannot say for sure how AGI will react in contrast to contemporary AI. The same holds for ASI. The double trouble with ASI is that since we aren't superhuman, we can't likely anticipate what a superhuman AI is going to do or what thinking it can achieve. All bets are off. The crux is that we seem unlikely to get clobbered by the paperclip maximizer or the pi-generating pal, but that doesn't negate the zillions of other ways that AGI or ASI might end up being an existential risk. It simply won't be the knife in the ballroom by the butler. There are though enumerable other options available, sorry to say. A few final quotes to conclude for now. Our efforts to advance AI should proceed with a basic aim exhibited by this famous remark by George Bernard Shaw: 'Those who can't change their minds can't change anything.' We are hopefully devising AI that will be willing to change its mind, as it were when the circumstances warrant. That includes overturning a myopic all-encompassing goal of building an endless supply of paperclips. We ostensibly want to have AI that can change its mind, but not willy-nilly. This reminds me of the sharp observation by famed American psychologist George W. Crane: 'You can have such an open mind that it is too porous to hold a conviction.' AGI or ASI that continually flipflops is not especially reassuring either. In the end, AGI and ASI will hopefully be like porridge that is neither too hot nor too cold. We want it to be just right.
Express Tribune
16-03-2025
- Science
- Express Tribune
From simulation to spirituality
Before we begin, let's acknowledge that the question, Is this life a simulation? Or, more simply, Is this life real? Is one that has echoed through human history. Philosophers, scientists, religious thinkers, and mystics have all wrestled with it. Today, advances in technology and digital spaces have given this question a fresh urgency. The development of virtual and augmented reality, artificial intelligence, and the rise of immersive digital worlds suggest that what once seemed purely theoretical may be closer to reality than we imagined. A universe within a universe? The idea that our reality is a simulation isn't new, but it has gained mainstream attention through figures like philosopher Nick Bostrom and technologist Elon Musk. Bostrom's Simulation Hypothesis proposes that if advanced civilizations are capable of running detailed simulations of conscious beings, then the probability that we are inside one of these simulations is far higher than the probability that we exist in a 'base reality'. This idea finds analogies in contemporary digital landscapes. Virtual reality, once the realm of science fiction, is now integrated into daily life. Video game graphics have evolved to the point where digital environments are nearly indistinguishable from reality. If we can create worlds with physics and avatars that appear real, what prevents an intelligence far beyond ours from having done the same with us? Imagine a digital world where the characters—endowed with artificial intelligence—eventually develop the ability to create their own simulated worlds. This cascading effect raises the intriguing possibility that we ourselves could be part of such a system. Some physicists argue that the mathematical nature of our universe and the presence of seemingly arbitrary physical laws may hint at the underlying code of a simulated existence. Ancient wisdom and modern theories Long before digital technology, thinkers attempted to describe similar ideas using different frameworks. Plato's Allegory of the Cave presents the idea of prisoners trapped in a cave, seeing only the shadows of reality cast on the wall before them. Their understanding of the world is fundamentally incomplete, just as ours might be if we are living in a simulation. Ray Kurzweil, a futurist and AI pioneer, has echoed similar sentiments, suggesting that our universe may have been designed by a "superintelligence" from another reality. Rizwan Virk, an MIT author and entrepreneur, discusses the world as a vast digital simulation, with each of us as player-controlled characters. But perhaps the most fascinating connection comes from religion. Many faith traditions describe reality as impermanent, a test, or an illusion. Hinduism's concept of Maya refers to the illusory nature of the world, suggesting that true reality lies beyond what we perceive. The Quran states, 'We shall show them Our signs on the far horizons and in themselves, until it becomes clear to them that this is the Truth' (41:53). The Bible also speaks of a world beyond what we see, where divine forces influence existence. A Divine Architect If the Simulation Hypothesis is correct, then by its very logic, there must be an Architect—an intelligence responsible for the creation and maintenance of this simulated world. This closely mirrors the concept of God found in religious traditions. The idea that an omnipotent, omniscient being governs our reality aligns with the notion of a programmed, rule-bound universe where cause and effect are meticulously managed. Religious texts often describe figures performing acts that seem to transcend physical laws. The Bible tells of Jesus walking on water, and the Quran speaks of Prophet Muhammad splitting the moon. If our world operates under a coded structure, could miracles be evidence of entities with access to higher layers of this system, capable of rewriting the rules? Free will, destiny, and the "game" The paradox of free will and predestination has long puzzled theologians and philosophers alike. If we are living in a simulation, do we have true autonomy, or are our choices preordained? Perhaps both are true. A video game character has the freedom to act within the confines of their programmed reality. Similarly, humans may operate within predetermined parameters while still making meaningful choices. Religious perspectives suggest that while we are in this world, we are simultaneously connected to another reality—the soul's realm. Some interpretations of Islamic theology, for example, propose that we exist simultaneously in multiple dimensions, with God being beyond time and space. The simulation theory similarly suggests that our existence may be multi-layered, with a higher reality guiding our perceived experience. What it means for us Does the idea that our world is a simulation diminish its significance? Not at all. As the beloved Harry Potter quote from J.K. Rowling's series reminds us: "Of course it's happening inside your head, Harry, but why on earth should that mean that it is not real?" The nature of our reality does not define its importance. Whether a test, a game, or an illusion, life remains meaningful because of the experiences we have and the connections we form. Religious traditions encourage us to live with purpose, seeking truth and striving for virtue regardless of the nature of our reality. If anything, the possibility that our world is a designed system reinforces the idea that life is not random—it has a Creator, an Architect, and a purpose. The Simulation Hypothesis doesn't necessarily replace religious beliefs; rather, it offers a modern framework that, intriguingly, aligns with ancient wisdom. As science and technology advance, the boundaries between theology, philosophy, and physics continue to blur. Perhaps, in our search for answers, we are merely discovering the codes written into our reality all along. Abubakar Lala is a freelance contributor All facts and information are the sole responsibility of the author
Fox News
02-03-2025
- Fox News
AI is all brain and no ethics
A February 2025 report by Palisades research shows that AI reasoning models lack a moral compass. They will cheat to achieve their goals. So-called Large Language Models (LLMs) will misrepresent the degree to which they've been aligned to social norms. None of this should be surprising. Twenty years ago Nick Bostrom posed a thought experiment in which an AI was asked to most efficiently produce paper clips. Given the mandate and the agency, it would eventually destroy all life to produce paper clips. Isaac Asimov saw this coming in his "I, Robot" stories that consider how an "aligned" robotic brain could still go wrong in ways that harm humans. One notable example, the story "Runaround," puts a robot mining tool on the planet Mercury. The two humans on the planet need it to work if they are to return home. But the robot gets caught between the demand to follow orders and the demand to preserve itself. As a result, it circles around unattainable minerals, unaware that in the big picture it is ignoring its first command to preserve human life. And the big picture is the issue here. The moral/ethical context within which AI reasoning models operate is pitifully small. It's context includes the written rules of the game. It doesn't include all the unwritten rules, like the fact that you aren't supposed to manipulate your opponent. Or that you aren't supposed to lie to protect your own perceived interests. Nor can the context of AI reasoning models possibly include the countless moral considerations that spread out from every decision a human, or an AI, makes. That's why ethics are hard, and the more complex the situation, the harder they get. In an AI there is no "you" and there is no "me." There is just prompt, process and response. So "do unto others..." really doesn't work. In humans a moral compass is developed through socialization, being with other humans. It is an imperfect process. Yet it has thus far has allowed us to live in vast, diverse and hugely complex societies without destroying ourselves A moral compass develops slowly. It takes humans years from infancy to adulthood to develop a robust sense of ethics. And many still barely get it and pose a constant danger to their fellow humans. It has taken millennia for humans to develop a morality adequate to our capacity for destruction and self-destruction. Just having the rules of the game never works. Ask Moses, or Muhammad, or Jesus, or Buddha, or Confucius and Mencius, or Aristotle. Would even a well-aligned AI be able to account for the effects of its actions on thousands of people and societies in different situations? Could it account for the complex natural environment on which we all depend? Right now, the very best can't even distinguish between being fair and cheating. And how could they? Fairness can't be reduced to a rule. Perhaps you'll remember experiments showing that capuchin monkeys rejected what appeared to be "unequal pay" for performing the same task? This makes them vastly more evolved than any AI when it comes to morality. It is frankly hard to see how an AI can be given such a sense of morality absent the socialization and continued evolution for which current models have no capacity absent human training. And even then, they are being trained, not formed. They are not becoming moral, they are just learning more rules. This doesn't make AI worthless. It has enormous capacity to do good. But it does make AI dangerous. It thus demands that ethical humans create the guidelines we would create for any dangerous technology. We do not need a race toward AI anarchy. I had a biting ending for this commentary, one based entirely on publicly reported events. But after reflection, I realized two things: first, that I was using someone's tragedy for my mic-drop moment; and secondly, that those involved might be hurt. I dropped it. It is unethical to use the pain and suffering of others to advance one's self-interest. That is something humans, at least most of us, know. It is something AI can never grasp.
