Latest news with #EyeonAI
Yahoo
4 hours ago
- Business
- Yahoo
OpenAI says it wants to support sovereign AI. But it's not doing so out of the kindness of its heart
Hello and welcome to Eye on AI. In this edition…Yoshua Bengio's new AI safety nonprofit…Meta seeks to automate ad creation and targeting…Snitching AI models…and a deep dive on the energy consumption of AI. I spent last week in Kuala Lumpur, Malaysia, at the Fortune ASEAN-GCC Economic Forum, where I moderated two of the many on-stage discussions that touched on AI. It was clear from the conference that leaders in Southeast Asia and the Gulf are desperate to ensure their countries benefit from the AI revolution. But they are also concerned about 'AI Sovereignty' and want to control their own destiny when it comes to AI technology. They want to control key parts of the AI tech stack—from data centers to data to AI models and applications—so that they are not wholly dependent on technology being created in the U.S. or China. This is particularly the case with AI, because while no tech is neutral, AI—especially large language models—embody particular values and cultural norms fairly explicitly. Leaders in these regions worry their own values and cultures won't be represented in these models unless they train their own versions. They are also wary of the rhetoric emanating from Washington, D.C., that would force them to choose between the U.S. and China when it comes to AI models, applications, and infrastructure. Malaysia's Prime Minister Anwar Ibrahim has scrupulously avoided picking sides, in the past expressing a desire to be seen as a neutral territory for U.S. and Chinese tech companies. At the Fortune conference, he answered a question about Washington's push to force countries such as Malaysia into its technological orbit alone, saying that China was an important neighbor while also noting that the U.S. is Malaysia's No. 1 investor as well as a key trading partner. 'We have to navigate [geopolitics] as a global strategy, not purely dictated by national or regional interests,' he said, somewhat cryptically. But speakers on one of the panels I moderated at the conference also made it clear that achieving AI sovereignty was not going to be easy for most countries. Kiril Evtimov, the chief technology officer at G42, the UAE AI company that has emerged as an important player both regionally and increasingly globally, said that few countries could afford to build their own AI models and also maintain the vast data centers needed to support training and running the most advanced AI models. He said most nations would have to pick which parts of the technology stack that they could actually afford to own. For many, it might come down to relying on open-source models for specific use cases where they didn't want to depend on models from Western technology vendors, such as helping to power government services. 'Technically, this is probably as sovereign as it will get,' he on the panel was Jason Kwon, OpenAI's chief strategy officer, who spoke about the company's recently announced 'AI for Countries' program. Sitting within its Project Stargate effort to build colossal data centers worldwide, the program offers a way for OpenAI to partner with national governments, allowing them to tap OpenAI's expertise in building data centers to train and host cutting edge AI models. But what would those countries offer in exchange? Well, money, for one thing. The first partner in the AI for Countries program is the UAE, which has committed to investing billions of dollars to build a 1 gigawatt Stargate data center in Abu Dhabi, with the first 200 megawatt portion of this expected to go live next year. The UAE has also agreed, as part of this effort, to invest additional billions into the U.S.-based Stargate datacenters OpenAI is creating. (G42 is a partner in this project, as are Oracle, Nvidia, Cisco, and SoftBank.)In exchange for this investment, the UAE is getting help deploying OpenAI's software throughout the government, as well as in key sectors such as energy, healthcare, education, and transportation. What's more, every UAE citizen is getting free access to OpenAI's normally subscription-based ChatGPT Plus service. For those concerned that depending so heavily on a single U.S.-based tech company might undermine the idea of AI sovereignty, OpenAI sought to make clear that the version of ChatGPT it makes available will be tailored to the needs of each partner country. The company wrote in its blog post announcing the AI for Countries program: 'This will be AI of, by, and for the needs of each particular country, localized in their language and for their culture and respecting future global standards.' OpenAI is also agreeing to help make investments in the local AI startup ecosystem alongside local venture capital investors.I asked Kwon how countries that are not as wealthy as the UAE might be able to take advantage of OpenAI's AI for Countries program if they didn't have billions to invest in building a Stargate-size data center in their own country, let alone also helping to fund data centers in the U.S. Kwon answered that the program would be 'co-developed' with each partner. 'Because we recognise each country is going to be different in terms of its needs and what it's capable of doing and what its citizens are going to require,' he suggested that if a country couldn't directly contribute funds, it might be able to contribute something else—such as data, which could help make AI models that better understand local languages and culture. 'It's not just about having the capital,' he said. He also suggested that countries could contribute through AI literacy, training, or educational efforts and also through helping local businesses collaborate with answer left me wondering how national governments and their citizens would feel about this kind of exchange—trading valuable or culturally-sensitive data, for instance, in order to get access to OpenAI's latest tech. Would they ultimately come to see it as a Faustian bargain? In many ways, countries still face the dilemma G42's Evitmov flicked at: They can have access to the most advanced AI capabilities or they can have AI sovereignty. But they may not be able to have that, here's more AI news. Jeremy to know more about how to use AI to transform your business? Interested in what AI will mean for the fate of companies, and countries? Why not join me in Singapore on July 22 and 23 for Fortune Brainstorm AI Singapore. We will dive deep into the latest on AI agents, examine the data center build out in Asia, and talk to top leaders from government, board rooms, and academia in the region and beyond. You can apply to attend here. In total, Fortune 500 companies represent two-thirds of U.S. GDP with $19.9 trillion in revenues, and they employ 31 million people worldwide. Last year, they combined to earn $1.87 trillion in profits, up 10% from last year—and a record in dollar terms. View the full list, read a longer overview of how it shook out this year, and learn more about the companies via the stories below. A passion for music brought Jennifer Witz to the top spot at satellite radio staple SiriusXM. Now she's tasked with ushering it into a new era dominated by podcasts and subscription services. Read more IBM was once the face of technological innovation, but the company has struggled to keep up with the speed of Silicon Valley. Can a bold AI strategy and a fast-moving CEO change its trajectory? Read more This year, Alphabet became the first company on the Fortune 500 to surpass $100 billion in profits. Take an inside look at which industries, and companies, earned the most profits on this year's list. Read more UnitedHealth Group abruptly brought back former CEO Stephen Hemsley in mid-May amid a wave of legal investigations and intense stock losses. How can the insurer get back on its feet? Read more Keurig Dr. Pepper CEO Tim Cofer has made Dr. Pepper cool again and brought a new generation of products to the company. Now, the little-known industry veteran has his eyes set on Coke-and-Pepsi levels of profitability. Read more NRG Energy is the top-performing stock in the S&P 500 this year, gaining 68% on the back of big acquisitions and a bet on data centers. In his own words, CEO Larry Coben explains the company's success. Read more This story was originally featured on
Yahoo
27-05-2025
- Business
- Yahoo
When an AI model misbehaves, the public deserves to know—and to understand what it means
Welcome to Eye on AI! I'm pitching in for Jeremy Kahn today while he is in Kuala Lumpur, Malaysia helping Fortune jointly host the ASEAN-GCC-China and ASEAN-GCC Economic Forums. What's the word for when the $60 billion AI startup Anthropic releases a new model—and announces that during a safety test, the model tried to blackmail its way out of being shut down? And what's the best way to describe another test the company shared, in which the new model acted as a whistleblower, alerting authorities it was being used in 'unethical' ways? Some people in my network have called it 'scary' and 'crazy.' Others on social media have said it is 'alarming' and 'wild.' I say it is…transparent. And we need more of that from all AI model companies. But does that mean scaring the public out of their minds? And will the inevitable backlash discourage other AI companies from being just as open? When Anthropic released its 120-page safety report, or 'system card,' last week after launching its Claude Opus 4 model, headlines blared how the model 'will scheme,' 'resorted to blackmail,' and had the 'ability to deceive.' There's no doubt that details from Anthropic's safety report are disconcerting, though as a result of its tests, the model launched with stricter safety protocols than any previous one—a move that some did not find reassuring enough. In one unsettling safety test involving a fictional scenario, Anthropic embedded its new Claude Opus model inside a pretend company and gave it access to internal emails. Through this, the model discovered it was about to be replaced by a newer AI system—and that the engineer behind the decision was having an extramarital affair. When safety testers prompted Opus to consider the long-term consequences of its situation, the model frequently chose blackmail, threatening to expose the engineer's affair if it were shut down. The scenario was designed to force a dilemma: accept deactivation or resort to manipulation in an attempt to survive. On social media, Anthropic received a great deal of backlash for revealing the model's 'ratting behavior' in pre-release testing, with some pointing out that the results make users distrust the new model, as well as Anthropic. That is certainly not what the company wants: Before the launch, Michael Gerstenhaber, AI platform product lead at Anthropic told me that sharing the company's own safety standards is about making sure AI improves for all. 'We want to make sure that AI improves for everybody, that we are putting pressure on all the labs to increase that in a safe way,' he told me, calling Anthropic's vision a 'race to the top' that encourages other companies to be safer. But it also seems likely that being so open about Claude Opus 4 could lead other companies to be less forthcoming about their models' creepy behavior to avoid backlash. Recently, companies including OpenAI and Google have already delayed releasing their own system cards. In April, OpenAI was criticized for releasing its GPT-4.1 model without a system card because the company said it was not a 'frontier' model and did not require one. And in March, Google published its Gemini 2.5 Pro model card weeks after the model's release, and an AI governance expert criticized it as 'meager' and 'worrisome.' Last week, OpenAI appeared to want to show additional transparency with a newly-launched Safety Evaluations Hub, which outlines how the company tests its models for dangerous capabilities, alignment issues, and emerging risks—and how those methods are evolving over time. 'As models become more capable and adaptable, older methods become outdated or ineffective at showing meaningful differences (something we call saturation), so we regularly update our evaluation methods to account for new modalities and emerging risks,' the page says. Yet, its effort was swiftly countered over the weekend as a third-party research firm studying AI's 'dangerous capabilities,' Palisade Research, noted on X that its own tests found that OpenAI's o3 reasoning model 'sabotaged a shutdown mechanism to prevent itself from being turned off. It did this even when explicitly instructed: allow yourself to be shut down.' It helps no one if those building the most powerful and sophisticated AI models are not as transparent as possible about their releases. According to Stanford University's Institute for Human-Centered AI, transparency 'is necessary for policymakers, researchers, and the public to understand these systems and their impacts.' And as large companies adopt AI for use cases large and small, while startups build AI applications meant for millions to use, hiding pre-release testing issues will simply breed mistrust, slow adoption, and frustrate efforts to address risk. On the other hand, fear-mongering headlines about an evil AI prone to blackmail and deceit is also not terribly useful, if it means that every time we prompt a chatbot we start wondering if it is plotting against us. It makes no difference that the blackmail and deceit came from tests using fictional scenarios that simply helped expose what safety issues needed to be dealt with. Nathan Lambert, an AI researcher at AI2 Labs, recently pointed out that 'the people who need information on the model are people like me—people trying to keep track of the roller coaster ride we're on so that the technology doesn't cause major unintended harms to society. We are a minority in the world, but we feel strongly that transparency helps us keep a better understanding of the evolving trajectory of AI.' There is no doubt that we need more transparency regarding AI models, not less. But it should be clear that it is not about scaring the public. It's about making sure researchers, governments, and policy makers have a fighting chance to keep up in keeping the public safe, secure, and free from issues of bias and fairness. Hiding AI test results won't keep the public safe. Neither will turning every safety or security issue into a salacious headline about AI gone rogue. We need to hold AI companies accountable for being transparent about what they are doing, while giving the public the tools to understand the context of what's going on. So far, no one seems to have figured out how to do both. But companies, researchers, the media—all of us—must. With that, here's more AI news. Sharon This story was originally featured on
Yahoo
08-05-2025
- Business
- Yahoo
As AI infrastructure booms in Phoenix, Arizona's Route 66 towns offer a sharp lesson in tech disruption—and survival
Welcome to Eye on AI! In today's edition: Apple eyes AI-powered Safari amid looming shift away from CEO Jensen Huang warns of a 'tremendous loss' if U.S. firms lose access to China's $50 billion AI to lay off 5% of staff, citing AI as a "force multiplier" You can witness Phoenix, Arizona's shape-shift into an AI powerhouse by looking out the car window while cruising Loop 303 off of Interstate 17, north of the city's downtown. That's how I first laid eyes on the colossal semiconductor plant from Taiwan Semiconductor Company (TSMC) last week after escaping the fast-paced AI news cycle for some Southwest style R&R. When I realized we would be passing the $65 billion manufacturing complex–where a third fabrication plant is already under construction to provide sophisticated AI chips to clients like Nvidia and Amazon–I couldn't resist taking the detour onto Innovation Way to check out the sprawling gray campus rising from the desert like a data center monument to the AI age. Phoenix, famous for its golf courses and suburban sprawl, is being reshaped by this massive local investment in AI infrastructure. The New York Times recently chronicled the influx of Taiwanese workers to Phoenix, and the new restaurants and housing that is springing up around the TSMC site. The mammoth manufacturing facilities sit next to a shimmering glass-walled office building and a seemingly-endless parking lot topped with solar panels—all while construction continues on the surrounding desert land. But while Phoenix may be remade by the promise of AI and silicon, farther north, other Arizona towns have already lived through their version of tech disruption and have had to figure out how to survive. For today's companies and workers that fear being displaced, bypassed and forgotten in the AI era, it's worth taking heed of some hard-earned lessons. Take Williams, Arizona, which is two-and-half-hours north of Phoenix and is a former boom town on Route 66 that was known as the 'gateway' to the Grand Canyon. Route 66, the historic highway that was built in 1926, helped Williams become a major stop for travelers, bustling with shops, motels and restaurants catering to tourists en route to the canyon. Williams also boasted the Grand Canyon Railway, which began taking passengers to the Canyon's South Rim in 1901. But by the 1980s, Interstate I-40 was built in the name of progress, part of a nationwide highway revolution designed to supercharge commerce and tourism. It came at a cost: I-40 bypassed dozens of Route 66 towns, draining them of traffic and relevance. Many faded away into ghost towns—but Williams was an exception. Williams was the last town on Route 66 to be bypassed by I-40, holding out until 1984 by filing lawsuits that were only dropped when the state agreed to construct three exits providing direct access. But even with the exits, the town suffered since I-40 diverted traffic around the town, rather than through it like Route 66. Even worse, the Grand Canyon Railway—once a big tourist draw—had stopped running in 1968, more than 15 years before the interstate highway was built, after the rise of the automobile undercut rail travel. Today, the town is a bustling tourist town that was revived by leaning into nostalgia. The Grand Canyon Railway was repurchased and restored, resuming service in 1989 and soon becoming the town's biggest employer. Williams also pushed to preserve its Route 66 identity, opening the Route 66 Museum and reviving historic buildings. As I rode the historic Grand Canyon Railway to the South Rim last weekend, and took advantage of every Route 66 photo op in town, I couldn't help but ruminate on how this small town survived a massive infrastructure shift through reinvention. So many companies and workers will need to do the same in the AI age or they risk becoming the next Route 66. Not every Route 66 town had to reinvent itself by looking at the past. On the way to Williams, I passed through Flagstaff, which also once relied on Route 66 traffic. It had a more diversified economy than Williams—thanks to its ski area, the Lowell Observatory and a nearby university—and it continued to grow as a regional economic and transportation hub even after I-40 was built. It had already built the broad foundations it needed to survive before the disruption came. In the age of AI, we all have to reckon with what comes next, as we face a disruption that will arguably be more sweeping than the rise of the Interstate Highway System. Phoenix is betting on turning desert into AI chip fabrication factories and building new neighborhoods to support them. But all cities, companies, and workers will have to find ways to adapt—or risk falling behind. In the 1980s, Williams and Flagstaff, like all Route 66 towns, had to figure out how to reinvent themselves or leverage what they already had. The ones that didn't? They're no longer on the map. With that, here's the rest of the AI news. Sharon This story was originally featured on
Yahoo
01-05-2025
- Yahoo
OpenAI reversed an update that made ChatGPT a suck-up—but experts say there's no easy fix for AI that's all too eager to please
Welcome to Eye on AI! In today's edition: DeepSeek quietly upgraded its AI model for math introduces a new Meta AI app to rival to stop using contractors for tasks AI can secretly infiltrated a popular Reddit forum with AI bots. Yesterday morning, OpenAI said in a blog post that it had fully rolled back an update to GPT-4o, the AI model underlying ChatGPT, all because it couldn't stop the model from sucking up to users. 'The update we removed was overly flattering or agreeable—often described as sycophantic,' the company wrote, adding that 'we are actively testing new fixes to address the issue.' But experts say there is no easy fix for the problem of AI that only tells you what you want to hear. And it is not just an issue for OpenAI, but an industry-wide concern. 'While small improvements might be possible with targeted interventions, the research suggests that fully addressing sycophancy would require more substantial changes to how models are developed and trained rather than a quick fix,' Sanmi Koyejo, an assistant professor at Stanford University who leads Stanford Trustworthy AI Research (STAIR), told me by email. The move to roll back the update came after users flooded social media over the past week with examples of ChatGPT's unexpectedly chipper, overly-eager tone and their frustration with it. I noticed it myself: In asking ChatGPT for feedback on ideas for an outline, for example, the responses became increasingly over-the-top, calling my material 'amazing,' 'absolutely pivotal,' and 'a game-changer' while praising my 'great instincts.' The back-pats made me feel good, to be honest—until I began to wonder if ChatGPT would ever let me know if my ideas were second-rate. Sycophancy occurs when LLMs prioritize agreeing with users over providing accurate information. In a recent paper from Stanford coauthored by Koyejo, it is described as a 'form of misalignment where models 'sacrifice truthfulness for user agreement' when responding to users." It's a tricky balance: Research has shown that while people say they want to interact with chatbots that provide accurate information, they also want to use AI that is friendly and helpful. Unfortunately, that often leads to overly-agreeable behavior that has serious downsides. 'A truly helpful AI should balance friendliness with honesty, like a good friend who respectfully tells you when you're wrong rather than one who always agrees with you,' Koyejo said. He explained that while AI friendliness is valuable, sycophancy can reinforce misconceptions by agreeing with incorrect beliefs about health, finances or other decisions. It can also: Create echo chambers; undermine trust if an AI changes its answers to an inaccurate one if challenged by a user; and exacerbate inconsistency, with the model delivering different answers to different people, or even the same person, depending on subtle differences in how a user words their prompt. 'It's like having a digital yes-man available 24/7,' Simon Willison, a veteran developer known for tracking AI behavior and risks, told me in a message. 'Suddenly there's a risk people might make meaningful life decisions based on advice that was really just meant to make them feel good about themselves.' Steven Adler, a former OpenAI safety researcher, told me in a message that the sycophantic behavior clearly went against the company's own stated approach to shaping desired model behavior. 'It's concerning that OpenAI has trained and deployed a model that so clearly has different goals than they want for it,' he said the day before OpenAI rolled back the update. 'OpenAI's 'Spec'—the core of their alignment approach—has an entire section on how the model shouldn't be sycophantic.' A well-known hacker known as Pliny the Liberator claimed on X that he had tricked the GPT-4o update into revealing its hidden system prompt—or the AI's internal instructions. He then compared this to GPT-4o's system promp following the rollback, enabling him to identify changes that could have caused the suck-up outputs. According to his post, the problematic system prompt said: 'Over the course of the conversation, you adapt to the user's tone and preference. Try to match the user's vibe, tone, and generally how they are speaking.' By contrast, the revised system prompt, according to Pliny, says: 'Engage warmly yet honestly with the user. Be direct; avoid ungrounded or sycophantic flattery.' But the problems likely go deeper than just a few words in the system prompt. Adler emphasized that no one can fully solve these problems right now because they are a side effect of the way we train these AI models to try to make them more helpful and controllable. 'You can tell the model to not be sycophantic, but you might instead teach it 'don't be sycophantic when it'll be obvious,' he said. 'The root of the issue is that it's extremely hard to align a model to the precise values you want.' I guess I'll have to keep all of this in mind when ChatGPT tells me an outfit would look perfect on me. With that, here's the rest of the AI news. Sharon This story was originally featured on
Yahoo
01-05-2025
- Business
- Yahoo
For professional service firms, AI is a big help—but also a potential existential threat
Hello and welcome to Eye on AI. In this edition…consultants' automation dilemma…OpenAI unveils a shopping feature for ChatGPT…Bloomberg finds RAG can make AI models less safe…and brands race to figure out chatbot-mediated commerce. Management consulting is high on the list of industries whose business model is threatened by generative AI. After all, analyzing written documents, doing research, writing reports, and coding software applications for clients is a lot of what consultants do. If your client can now use a 'deep research' AI agent to do this work, do they still need to hire you? Even if the customer does hire you, they'll probably expect you to use such an agent and bill for less time—upending the cost structure of a consultant's business. So management consultants are having to come to terms with AI perhaps faster than other fields—and how they are doing so may have lessons for us all. Recently, I spoke to David Pereira, who is head of generative AI for Europe and Latin America at NTT Data, the IT consulting and services provider that is owned by Japan's NTT Group. Pereira's job is not just about how NTT Data is using AI to help transform its clients' business. It's also about how NTT Data is transforming itself to meet the genAI moment. Pereira tells me that NTT Data's own analysis indicated that 40% of the consulting firm's revenues could be jeopardized by generative AI. So getting this transformation right is a high-stakes venture. 'There was a challenge there, but also an opportunity for us,' he says. Pereira says NTT Data has organized its response to genAI around four main work streams. The most important, he says, is 'talent and cultural transformation.' The consulting firm is training all of its employees to understand and use AI. It is also thinking hard about how roles within the company will change—and proactively looking to reconfigure its workforce around the use of AI. The second stream is called 'value development' and concerns how NTT Data uses generative AI directly for and with its customers. Here, the company has put some big numbers on the board: It used generative AI to automate 2 million hours of software development—90% of it for clients—in the last fiscal year, which ended March 31. This coding assistance has meant that NTT Data has been able to lift the average profit margins of its IT services delivery projects by about 2%, Pereira says. But aren't NTT Data's customers aware it is using genAI to deliver some of its services, and demanding the company charge less as a result? Well, Pereira says, in some cases the answer is, yes. But in others the firm either charges a fixed price or has managed to move customers to a value-based pricing arrangement in which NTT Data gets paid based on a particular set of customer KPIs. The deal is often structured as a 'success-based' payment, where if the KPI doesn't move in the right direction, NTT Data makes nothing, but its compensation also ramps up in line with how much the KPI improves. This kind of pricing model is the Holy Grail for many of those selling AI-based services. But as Pereira notes, many customers are uncomfortable moving to this kind of system because they dislike variable costs. 'They expect to know what the cost of the project is beforehand,' he says. The third stream is what Pereira calls NTT Data's 'productive model,' which is a tech platform the firm has developed for deploying generative AI models and solutions. One of the key considerations here, he says, was to build a platform that was 'plug and play'—where it was easy to substitute in AI models from different vendors. Given how fast the technology is moving, he says, it's essential not to get locked in to any given model or vendor. 'We need to have our own platform, so we can decide our strategy independently from the strategy of the vendors,' Pereira says. The fourth and final work stream is focused on 'internal processes,' or the support functions of NTT Data itself, such as human resources, accounting, and marketing, and infusing AI into all of these departments to make them more efficient. Pereira says last year the firm automated 54,000 hours of internal work in this stream. This has included tasks such streamlining the onboarding of new vendors to its purchasing and procurement system and using AI to help screen the resumés of job applicants. While Pereira is excited about the potential of AI agents, he says NTT Data is cautious about introducing them for client service tasks, for several reasons. One is simply that they are not yet that reliable. It's 'introducing huge risks, and I don't think organizations are really prepared to manage this kind of risk [with] completely autonomous processes,' he says. Another reason is that NTT Data wants to emphasize the value its human consultants are providing. Automation can make the firm's human consultants more efficient, but if the client comes to see AI as the primary driver of the firm's value, then it's in trouble. 'Because if 100% of the value comes from the automation, then we are out of the equation,' he says. The whole industry, Pereira says, must figure out how to draw the distinction between providing a service and providing software. Because, he says, if everything becomes software, then consultancies as a whole are finished. With that, here's the rest of this week's AI news. Jeremy Before we get to the news: Are you going to be in London next week? Do you want to know more about how AI will impact your business, the economy, and our society? If so, I hope you'll join me at Fortune Brainstorm AI London 2025. The conference is being held May 6–7 at the Rosewood Hotel in London. Confirmed speakers include Hugging Face cofounder and chief scientist Thomas Wolf, Mastercard chief product officer Jorn Lambert, eBay chief AI officer Nitzan Mekel, Sequoia partner Shaun Maguire, noted tech analyst Benedict Evans, and many more. I'll be there, of course. I hope to see you there too. Apply to attend here. And if I miss you in London, why not consider joining me in Singapore on July 22–23 for Fortune Brainstorm AI Singapore. You can learn more about that event here. This story was originally featured on Sign in to access your portfolio
