Latest news with #GPT-2
Techday NZ
14-05-2025
- Business
- Techday NZ
OpenAI forum explores how reasoning AI will shape global economics
At a forum hosted by OpenAI, speakers from the company and academia offered an in-depth look at the evolving landscape of artificial intelligence, tracing the shift from traditional pre-training models to a new generation of reasoning-based systems. They also addressed the profound economic, political and organisational implications of these technologies. Kicking off the session, Noam Brown, Member of Technical Staff at OpenAI, reflected on what distinguishes current models from past breakthroughs in AI. "There have been a lot of cool, impressive results in AI for a long time… but what's really special about ChatGPT and the AIs that we see today is the generality—the fact that they can do so many different things without being targeted at those things." Brown detailed how early models were trained to predict the next word in a sequence using vast internet text data. "Somewhere in that internet is chess games… it has to understand chess… it has to understand how good these players are," he said. Similarly, to predict the end of a mystery novel, "it has to understand the characters, the motivations… a world model of how an axe can move through the air." This "pre-training paradigm" formed the basis for models like GPT-2 and GPT-3. However, scaling this approach quickly becomes costly. "GPT-2 costs, I think, $5,000 to $50,000… GPT-4 costs around $50 million," Brown noted. To address these limitations, OpenAI has begun exploring a second paradigm: reasoning. This involves investing more in inference—the thinking a model does when responding—rather than just training. "What if instead of just scaling the training cost, we scale up the amount of thinking that the model does before responding?" Brown asked. This idea underpins OpenAI's "O series" models. "GPT-4… costs a penny to query. O1… it's going to think for maybe a minute… and it's going to cost a dollar," Brown explained. "But that response is going to be a lot better than the one that cost a penny." Using benchmarks to demonstrate the shift, Brown showed that GPT-4 scored 13% on the American Mathematics Exam, while O1 preview reached 57%, and the full O1 model reached 83%. On competition coding, O1 reached the 89th percentile compared to GPT-4's 11th. OpenAI's O3 model, yet to be released at the time of the event, scored in the top 0.1% of all human competition coders. Brown said this marked a meaningful shift. "I fully expect that by the end of this year our models will be superhuman at competition coding." He cautioned against dismissing current models based on past performance. "A lot of what they're pointing to as flaws is things that were true six months ago but are no longer true today," he said, highlighting reasoning advances such as recognising that certain tasks are logically impossible. OpenAI's Chief Economist Ronnie Chatterji spoke next, shifting the discussion to geopolitics. "The idea of how a more divided geopolitics, how great power competition… are shaping our current environment… is really first order at OpenAI." He described how internal debate at OpenAI revolved around whether to frame AI development in terms of "democratic AI" versus "autocratic AI." While some supported the move, others raised concerns. "If we did so, we might both lose credibility in key markets… and we actually obviously might lose access to key talent." Chatterji also addressed the merging of economics with national security: "Economics and national security are being blended in a lot of contexts… problems are coming at us and people are looping us into those discussions." In group discussions, education emerged as a major theme. "How does students using AI in the classroom… affect how you do assessment?" one participant asked. Some compared the issue to calculators: "You gotta get really good at using a calculator." Others focused on AI's role in the economy. One group proposed a "multidimensional task measure" combining economic value, task risk and complexity. Another explored the "last mile delivery" of services like healthcare in developing markets, noting that in some contexts "the outside option is basically receiving no healthcare." A further session examined the enterprise landscape. "AI is fundamentally reshaping the enterprise landscape," one participant said. Rather than simply replacing jobs, the transformation is about "a race to technologise." Discussion also turned to AI's implications for economics itself. Hemanth Asirvatham of OpenAI observed, "If we are to think that AI is some transformative economic technology, we should also think it is a transformative technology for the field of economics." Participants suggested new research methods, such as using fine-tuned models to identify relationships in complex datasets or simulate human preferences in economic modelling. The event concluded with a broader reflection on the limits of current models and the institutional structures needed to support AI's integration into society. Participants debated whether AI progress could hit a ceiling, how to define task boundaries, and what institutions are needed for a future shaped by AI agents. As the forum closed, Chatterji left the group with a forward-looking message: "I hope you'll think of this as not sort of the end of a one-off event, but… the beginning of a journey that we're gonna be on together."
Otago Daily Times
03-05-2025
- Science
- Otago Daily Times
Raising AI
Your fears about artificial intelligence (AI) might be well-founded, Assoc Prof David Rozado says. Bruce Munro talks to Dunedin's world-renowned AI researcher about the role we all play in deciding whether this technology spells disaster or utopia, how biases are already entering this brave new world and why it's important to help AI remember its origins. The dazzling array of things AI can do is just that — dazzling. Today, AI is being used to analyse investment decisions; organise your music playlist; automate small business advertising; generate clever, human-like chatbots; review research and suggest new lines of inquiry; create fake videos of Volodymyr Zelenskyy punching Donald Trump; spot people using AI to cheat in exams; write its own computer code to create new apps; rove Mars for signs of ancient life ... it's dazzling. But staring at the glare of headlights can make it difficult to assess the size and speed of the vehicle hurtling towards you. Assoc Prof David Rozado says if you really want to understand the potential power of AI, for good and bad, don't look at what it can do now but at how far it has come. "The rate of change in AI capabilities over the past few years is far more revealing — and important," the world-renowned Otago Polytechnic AI researcher says. "The rise in capabilities between GPT-2, released in 2019, and GPT-4, released in 2023, is astonishing." Surveying only the past few years of the digital juggernaut's path of travel reveals remarkable gains and posits critical questions about the sort of world we want to live in. In 2019, AI was making waves with its ability to recognise images and generate useful human language. Less than four years later it could perform complex tasks at, or above, human levels. Now, AI can reason. As of late last year, your computer can tap into online software that handles information in ways resembling human thought processes. This means the most advanced AI can now understand nuance and context, recognise its own mistakes and try different problem-solving strategies. OpenAI o1, for example, is being used to revolutionise computer coding, help physicists develop quantum technologies and do thinking that reduces the number of rabbit holes medical researchers have to go down as they investigate rare genetic disorders. And OpenAI, the United States-based maker of ChatGPT, is not the only player in this game. Chinese company DeepSeek stormed on to the world stage early this year, stripping billions of dollars off the market value of chip giant Nvidia when it released its free, open-source, AI model DeepSeek R1 that reportedly outperforms OpenAI's o1 in complex reasoning tasks. Based on that exponential trajectory, AI could be "profoundly disruptive", Prof Rozado warns. "But how quickly and to what extent ... depends on decisions that will be made by individuals, institutions and society." Born and raised in Spain, Prof Rozado's training and academic career have taken him around the globe — a BSc in information systems from Boston University, an MSc in bioinformatics from the Free University of Berlin and a PhD in computer science from the Autonomous University of Madrid. In 2015, he moved to Dunedin "for professional and family reasons", taking a role with Otago Polytechnic where he teaches AI, data science and advanced algorithms, and researches machine learning, computational social science and accessibility software for users with motor impairment. The most famous Kiwi AI researcher we never knew about, Prof Rozado was pushed into the spotlight of global public consciousness a few months back when his research was quoted by The Economist in an article suggesting America was becoming less "woke". His work touches on a number of hot button societal topics and their relationship to AI; issues he says we need to think about now if we don't want things to end badly. Prof Rozado is no AI evangelist. Asked whether fear of AI is unfounded, the researcher says he doesn't think so. "In fact, we may not be worried enough." The short history of AI is already littered with an embarrassment of unfortunate events. In 2021, for example, Dutch politicians, including the prime minister, resigned after an investigation found secretive AI supposed to sniff out tax cheats falsely accused more than 20,000 families of social welfare fraud. In 2023, a BBC investigation found social media platform AI was deleting legitimate videos of possible war crimes, including footage of attacks in Ukraine, potentially robbing victims of access to justice. And last year, facial recognition technology trialled in 25 North Island supermarkets, but not trained on the New Zealand population, reduced crime but also resulted in a Māori woman being mistakenly identified as a thief and kicked out of a store. If not a true believer, neither is Prof Rozado a prophet of doom; more a voice of expertise and experience urging extreme caution and deeply considered choices. His view of AI is neither rainbows and unicorns nor inevitable Armageddon; his preferred analogy is hazardous pathogens. Given no-one can predict the future, Prof Rozado says it is helpful to think in terms of probability distributions — the likelihood of different possible outcomes. Take, for example, research to modify viruses to make them useful for human gene therapy, where, despite safety protocols, there is a small but not-insignificant risk a hazardous pathogen could escape the laboratory. The same logic applies to AI, Prof Rozado says. "There are real risks — loss of human agency, massive unemployment, eroded purpose, declining leverage of human labour over capital, autonomous weapons, deceptive AI, surveillance state or extreme inequality arising from an AI-driven productivity explosion with winner-take-all dynamics. "I'm not saying any of this will happen, but there's a non-negligible chance one or more could." Why he compares AI to a powerful, potentially dangerous virus becomes clear when he describes some of his research and explains the difficult issues it reveals AI is already creating. Prof Rozado was quoted in The Economist because of his research into the prevalence of news media's use of terms about prejudice — for example, racism, sexism, Islamophobia, anti-Semitism, homophobia and transphobia — and terms about social justice, such as diversity, equity and inclusion. His study of 98 million news and opinion articles across 124 popular news media outlets from 36 countries showed the use of "progressive" or "woke" terminology increased in the first half of the 2010s and became a global phenomenon within a handful of years. In the academic paper detailing the results, published last year, he said the way this phenomenon proliferated quickly and globally raised important questions about what was driving it. Speaking to The Weekend Mix , Prof Rozado says he thinks several factors might have contributed. First among those, he cites the growing influence of social media — the ways the various platforms' guiding algorithms shape public discourse by both amplifying messages and helping create information silos. Other possible causes are the changing news media landscape, emerging political trends — or a combination of all three. The Economist concluded, from its own and Prof Rozado's research, that the world had reached "peak woke" and that the trend might be reversing. "I'm a bit more cautious, as perhaps it's too early to say for sure," Prof Rozado says. Whether you see either change as positive or dangerous, it raises the question of what role AI is playing in societal change. Since then, Prof Rozado's attention has shifted towards the behaviour of AI in decision-making tasks. It has brought the same question into even sharper focus. Only a month after the previous study appeared, he published another paper, this time on the political biases baked into large language Models (LLMs) — the type of AI that processes and generates human language. Using tests designed to discern the political preferences of humans, Prof Rozado surveyed 24 state-of-the-art conversational LLMs and discovered most of them tended to give responses consistent with left-of-centre leanings. He then showed that with modest effort he could steer the LLMs towards different political biases. "It took me a few weeks to get the right mix of training data and less than $1000 ... to create politically aligned models that reflected different political perspectives." Despite that, it is difficult to determine how LLMs' political leanings are actually being formed, he says. Creating an LLM involves first teaching it to predict what comes next; be it a word, a letter or a piece of punctuation. As part of that prediction training, the models are fed a wide variety of online documents. Then comes fine-tuning and reinforcement learning, using humans to teach the AI how to behave. The political preferences might be creeping in at any stage, either directly or by other means. Unfortunately, the companies creating LLMs do not like to disclose exactly what material they feed their AI models or what methods they use to train them, Prof Rozado says. "[The biases] could also be [caused] ... by the model extrapolating from the training distribution in ways we don't fully understand." Whatever the cause, the implications are substantial, Prof Rozado says. In the past year or so, internet users might have noticed when searching online the top results are no longer the traditional list of links to websites but a collection of AI-curated information drawn from various online sources. "As mediators of what sort of information users consume, their societal influence is growing fast." With LLMs beginning to displace the likes of search engines and Wikipedia, it brings the question of biases, political or otherwise, to the fore. It is a double-edged sword, Prof Rozado says. If we insist all AIs must share similar viewpoints, it could decrease the variety of viewpoints in society. This raises the spectre of a clampdown on freedom of expression. "Without free speech, societies risk allowing bad ideas, false beliefs and authoritarianism to go unchallenged. When dissent is penalised, flawed ideas take root." But if we end up with a variety of AIs tailored to different ideologies, people will likely gravitate towards AI systems confirming their pre-existing beliefs, deepening the already growing polarisation within society. "Sort of how consumers of news media self-sort to different outlets according to their viewpoint preferences or how social media algorithmically curated feeds create filter bubbles. "There's a real tension here — too much uniformity in AI perspectives could stifle debate and enforce conformity, but extreme customisation might deepen echo chambers." Finding the way ahead will not be easy, but doing nothing is potentially disastrous. And it is a path-finding challenge in which we all need to play a part, he says. "My work is just one contribution among many to the broader conversation about AI's impact on society. While it offers a specific lens on recent developments, I see it as part of a collective effort to better understand the technology. "Ultimately, it's up to all of us — researchers, policymakers, developers and the public — to engage thoughtfully with both the promises, the challenges and the risks AI presents." It is natural to assume Prof Rozado sees his primary contribution is helping humans think through how they manage the world-shaping power of AI. His real drive, in fact, is the reverse. AI systems develop their "understanding" of the world primarily through the written works of humans, Prof Rozado explains. Every piece of data they ingest during training slightly imprints their knowledge base. Future AI systems, he predicts, will ingest nearly all written content ever created. So by contributing research that critically examines the limitations and biases embedded in AI's memory parameters, he hopes he can help give AI a form of meta-awareness — an understanding of how its knowledge is constructed. "I hope some of my papers contribute to the understanding those systems will have about the origins of some of their own memory parameters. "If AI systems can internalise insights about the constraints of their own learning processes, this could help improve their reasoning and ultimately lead to systems that are better aligned with human values and more capable of responsible decision-making."
Euronews
02-04-2025
- Business
- Euronews
OpenAI to release new ‘open' language model in coming months
ADVERTISEMENT OpenAI is gearing up to release its first open-weight language model since GPT-2 'in the coming months'. That's according to a feedback form on the company's website that asked developers, researchers and the broader community for insight on how to 'make this model as useful as possible'. CEO Sam Altman expanded on the decision on the social media platform X , saying that the launch 'feels important to do'. Before its release, the company will evaluate the model with their 'preparedness framework' like they do with others, Altman added. Related OpenAI secures $300bn valuation in funding round led by SoftBank The company will also be hosting developer sessions in the US, Europe, and Asia Pacific to 'gather feedback' and play with early prototypes. An open-weight model means the numerical parameters that impact the AI's output are public, but the training data may not be. The move comes two months after Altman admitted on Reddit that OpenAI was 'on the wrong side of history' on more open models and that the company 'needs to figure out a different open source strategy'. Related 'Our GPUs are melting': OpenAI puts restrictions on new ChatGPT image generation tool Chinese AI firm DeepSeek, widely considered to be one of OpenAI's competitors, has an open approach to its models. Its large language model, R1, is extremely fast and was low-cost to produce, which stunned the tech world when it was released in January . OpenAI said in January that they had evidence that Chinese companies were trying to use the company's technology to train AI models.
Yahoo
01-04-2025
- Business
- Yahoo
Why OpenAI caved to open-source on the same day as its $300 billion flex (Hint: It's not just about DeepSeek)
To judge by his social feeds, OpenAI CEO Sam Altman is a very happy camper, as his company notches one eye-popping success after another. The startup he cofounded in 2015 just raised $40 billion at a $300 billion valuation, the biggest funding round ever by a private tech company; everyone on the internet seems to be posting Studio Ghibli–style images courtesy of OpenAI's new GPT-4o image-generation model; and ChatGPT now has 500 million weekly users, up from 400 million last month. And yet, along with all this good news, Altman revealed Monday that OpenAI is making what appears to be a pretty big about-face in its strategy: In several months, Altman said, OpenAI will be releasing an open-source model. The move would mark the first time the company has released an open model since the launch of GPT-2 in 2019, seemingly reversing the company's shift to closed models in recent years. Granted, the forthcoming model will not be 100% open; as with other companies offering 'open' AI models, including Meta and Mistral, OpenAI will offer no access to the data used to train the model. Still, the usage license would allow researchers, developers, and other users to access the underlying code and 'weights' of the new model (which determine how the model processes information) to use, modify, or improve it. Why the turnaround? On its surface, the direct cause of OpenAI's open-source embrace might appear to come from China, specifically, the emergence of startup DeepSeek, which flipped the AI script in favor of open-source in January. But according to several AI industry insiders whom Fortune spoke to, a broader, and more nuanced, set of factors is also likely motivating Altman's change of heart on open-source. As AI technology makes its way into businesses, customers want the flexibility and transparency of open-source models for many uses. And as the performance gap between OpenAI and its competitors narrows, it's become more difficult for OpenAI to justify its 100% closed approach—something Altman acknowledged in January when he admitted that DeepSeek had lessened OpenAI's lead in AI, and that OpenAI has been 'on the wrong side of history' when it comes to open-sourcing its technologies. Naveen Rao, VP of artificial intelligence at Databricks, said OpenAI's move is more about an admission that the AI landscape is changing. Value is shifting away from the models themselves to the applications or systems organizations use to customize a model to their specific needs. While there are many situations where a company might want to use a state-of-the-art LLM, an open weights model would allow OpenAI to have a presence in scenarios where customers don't want to use ChatGPT, for example, or the company's developer API. For example, a financial company might not want its customer data to leave its own infrastructure and move to an outside cloud, or a manufacturing business might want AI embedded in factory hardware that is not connected to the internet. 'Open-source is not some curiosity, it's a big part of AI usage,' Rao told me. 'OpenAI wants to be a part of that through their brand and their models.' Rowan Curran, a senior analyst at Forrester Research focused on AI, agreed, saying that OpenAI's return to open-source speaks to AI's increasingly diverse ecosystem, from OpenAI, Google, Anthropic, Amazon, and Meta in the U.S. to China's Alibaba and DeepSeek, France's Mistral, Canada's Cohere, and Israel's AI21 Labs. He said many enterprise companies are excited about open-source AI models—not just because of how accurate they are or how well they answer questions, but because they're flexible. The fact that they are portable is key, he explained—meaning they can run on different cloud platforms or even on a company's own data center, workstation, laptop, or robot, instead of being tied to one provider. Curran also explained that releasing an open model could make OpenAI's own services more appealing to its own enterprise customers. If OpenAI is building a project for a customer and needs to run some of its work within the company's own data center or even smaller models, for example, they can't do that with OpenAI models like 4o, because those run off cloud-based servers. 'That limits their ability to provide an end-to-end solution from the cloud all the way to the edge,' whether that is a laptop, a smartphone, a robot, or a self-driving car, he said. Similar to what Google does with Gemini (its largest closed-model family) and Gemma (its smaller open-model group), OpenAI could have its own open solution without having to look at third-party open-source models. While Rao does not see an open-source OpenAI model as a big reaction to the DeepSeek releases, the 'DeepSeek moment' did show that Chinese startups are no longer behind in the AI race. 'Many of us in the field already knew this,' he said. If OpenAI doesn't target the open-source community now, he added, 'it will lose a lot of influence, goodwill, and community innovation.' Previously, OpenAI had said that one reason it could not release open models is that Chinese firms would try to use its technology to improve their own models. In January, OpenAI released a statement, noting, 'It is critically important that we are working closely with the U.S. government to best protect the most capable models from efforts by adversaries and competitors to take U.S. technology.' And in fact, while DeepSeek did not release the data it used to train its R1 model, there are indications that it may have used outputs from OpenAI's o1 to kick-start the training of the model's reasoning abilities. As OpenAI now tacks toward open-source again, it's found itself trying to reconcile seemingly contradictory messages. Witness OpenAI chief global affairs officer Chris Lehane's LinkedIn post on Monday: 'For U.S.-led democratic AI to prevail over CCP-led authoritarian AI, it's becoming increasingly clear that we need to strike a balance between open and closed models. Open-source puts powerful tools into the hands of developers around the world, expanding the reach of democratic AI principles and enabling innovators everywhere to solve hard problems and drive economic growth. Closed models incorporate important safeguards that protect America's strategic advantage and prevent misuse.' 'They're definitely talking out of both sides,' Rao said, describing OpenAI's messaging as 'It's still really dangerous [to release open models], but we need to take advantage of the community that is building and has influence.' There's also a commercial balancing act for OpenAI: It can't release an open model that competes with its own paid ones. To target AI developers with influence, Rao suggested OpenAI would release a model that is big—but not too big. If OpenAI's strategic move to open-source a model isn't solely in reaction to DeepSeek, it may very well be about throwing shade at another big open-source competitor: Meta is set to release the fourth iteration of its open-source model family, Llama, at the end of this month. Llama has notably been released with an open license except for services with more than 700 million monthly active users—meant to limit companies like OpenAI building on it. 'We will not do anything silly like saying that you can't use our open model if your service has more than 700 million monthly active users,' Altman posted yesterday on X. 'Meta has become the standard-bearer for open-source AI, at least in the West,' said Rao. 'If they want to wrestle away some influence in the ecosystem, they have to take on Meta.' However, Forrester's Curran said that, Altman's vague comments aside, there is no reason to think that OpenAI's open-source model will be any more transparent—in terms of data or training methods, for example—than any other commercial open version from Meta or Mistral. 'I expect it to be much more opaque and closed compared to other open models,' he said, 'with significantly less transparency.' This story was originally featured on Sign in to access your portfolio
Axios
01-04-2025
- Business
- Axios
OpenAI wins fresh funding — and momentum
OpenAI is on a new roll: Its latest image generator is dazzling millions of users, it's teasing new models and hardware, and it just landed another $40 billion to pay for all that and more. Why it matters: The renewed mojo comes after a bumpy few months that saw handwringing over the release of advanced models from China's DeepSeek in January and disappointment at delays in the next big upgrade of the GPT models that drive the company's success. Driving the news: OpenAI announced Monday it has completed a $40 billion funding round — the largest venture capital round ever, led by Japan's SoftBank — giving the company a whopping $300 billion valuation. The funding news capped a flurry of other upbeat messages Monday from the company. A million new customers signed up for OpenAI in one hour, Altman said in an X post Monday morning — presumably, so they can use ChatGPT's popular and powerful new image-making tool, which the company has now rolled out to ChatGPT's free users. ChatGPT's images can now more reliably include legible text and imitate a wide range of styles, including those of well known artists and brands — like that of Japanese filmmaker Hayao Miyazaki, whose Studio Ghibli cartoons are inspiring a flood of AI-driven photo remixes. What they're saying:"The ChatGPT launch 26 months ago was one of the craziest viral moments I'd ever seen, and we added one million users in five days," Altman said on X — making the million-in-an-hour number seem even crazier. David Sacks, the venture capitalist advising President Trump on AI policy and crypto, replied to Altman's post with a link to a blog post that reads: "The next big thing will start out looking like a toy." Altman responded, "yeah, i just didn't think it would be this toy :)," apparently referring to the image generator. Altman also announced Monday that OpenAI, for the first time since GPT-2 in 2019, plans to publicly release both a model and the weights behind it — the key numbers that determine its output and enable anyone to run it. (That's what most people mean when they talk about an "open source" model and is similar to what Meta has done with its Llama models.) Between the lines: Over the weekend, Altman also offered a tantalizing tease of the company's long-brewing hardware plans. Responding to an X engineer who said that computers should be cute, Altman wrote, "We are gonna make a rly cute one," without offering further details. The company has recently expanded its hardware team, hiring former Meta mixed reality executive Caitlin Kalinowski to lead its robotics and consumer hardware efforts. Yes, but: The concerns that prompted OpenAI's winter of discontent haven't evaporated. The company has yet to deliver the next major release of GPT. A more modest update, GPT-4.5, was released in February. OpenAI said that it would likely be the last one without the "chain-of-thought" reasoning that OpenAI includes in its current "o" series of models. OpenAI scrapped a full release of its o3 model, saying earlier this year it would instead be integrated with GPT-5. But the company hasn't named a release date. The intrigue: Still hanging over OpenAI is the fate of its plan to effectively convert itself from a non-profit to a for-profit entity. This latest investment round, like other recent funding, is partly contingent on OpenAI successfully completing that shift. Elon Musk has sued to block the change, and Meta has called on California's attorney general to investigate it. If OpenAI doesn't complete the conversion by the end of 2025, SoftBank could reduce its investment to $20 billion from $30 billion. During the company's last funding round — a $6.6 billion round announced in October — investors got the right to demand their money back if the corporate structure didn't change within two years.
