New AI benchmarks test speed of running AI applications
SAN FRANCISCO, April 2 (Reuters) - Artificial intelligence group MLCommons unveiled two new benchmarks that it said can help determine how quickly top-of-the-line hardware and software can run AI applications.
Since the launch of OpenAI's ChatGPT over two years ago, chip companies have begun to shift their focus to making hardware that can efficiently run the code that allows millions of people to use AI tools. As the underlying models must respond to many more queries to power AI applications such as chatbots and search engines, MLCommons developed two new versions of its MLPerf benchmarks to gauge speed.
One of the new benchmarks is based on Meta's (META.O), opens new tab so-called Llama 3.1 405-billion-parameter AI model, and the test targets general question answering, math and code generation. The new format tests a system's ability to process large queries and synthesize data from multiple sources.
Nvidia (NVDA.O), opens new tab submitted several of its chips for the benchmark, and so did system builders such as Dell Technologies (DELL.N), opens new tab. There were no Advanced Micro Devices (AMD.O), opens new tab submissions for the large 405-billion-parameter benchmark, according to data provided by MLCommons.
For the new test, Nvidia's latest generation of artificial intelligence servers - called Grace Blackwell, which have 72 Nvidia graphics processing units (GPUs) inside - was 2.8 to 3.4 times faster than the previous generation, even when only using eight GPUs in the newer server to create a direct comparison to the older model, the company said at a briefing on Tuesday.
Nvidia has been working to speed up the connections of chips inside its servers, which is important in AI work where a chatbot runs on multiple chips at once.
The second benchmark is also based on an open-source AI model built by Meta and the test aims to more closely simulate the performance expectations set by consumer AI applications such as ChatGPT.
The goal is to tighten the response time for the benchmark and make it close to an instant response.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Independent
16 minutes ago
- The Independent
Reddit sues AI company Anthropic for allegedly 'scraping' user comments to train chatbot Claude
Social media platform Reddit has sued the artificial intelligence company Anthropic, alleging that it is illegally "scraping" the comments of Reddit users to train its chatbot Claude. Reddit claims that Anthropic has used automated bots to access Reddit's content despite being asked not to do so, and 'intentionally trained on the personal data of Reddit users without ever requesting their consent.' Anthropic didn't immediately return a request for comment Wednesday. The claim was filed Wednesday in the Superior Court of California in San Francisco. 'AI companies should not be allowed to scrape information and content from people without clear limitations on how they can use that data,' said Ben Lee, Reddit's chief legal officer, in a statement Wednesday. Reddit has previously entered licensing agreements with Google, OpenAI and other companies to enable them to train their AI systems on Reddit commentary. Those agreements 'enable us to enforce meaningful protections for our users, including the right to delete your content, user privacy protections, and preventing users from being spammed using this content,' Lee said.
Reuters
an hour ago
- Reuters
AI-driven search ad spending set to surge to $26 billion by 2029, data shows
June 4 (Reuters) - Spending on AI-powered search advertising is poised to surge to nearly $26 billion by 2029 from just over $1 billion this year in the U.S., driven by rapid adoption of the technology and more sophisticated user targeting, data from Emarketer showed on Wednesday. Companies that rely on traditional keyword-based search ads could experience revenue declines due to the growing popularity of AI search ads, which offer greater convenience and engagement for users, according to the research firm. Search giants such as Alphabet-owned (GOOGL.O), opens new tab Google and Microsoft's (MSFT.O), opens new tab Bing have added AI capabilities to better compete with chatbots such as OpenAI's ChatGPT and Perplexity AI, which provide users with direct information without requiring to click through multiple results. Apple (AAPL.O), opens new tab is exploring the integration of AI-driven search capabilities into its Safari browser, potentially moving away from its longstanding partnership with Google. The report has come as concerns grew about users increasingly turning to the chatbots for conversational search and AI-powered search results could upend business models of some companies. Online education firm Chegg (CHGG.N), opens new tab said in May that it would lay off about 248 employees as it looks to cut costs and streamline operations because students are using AI-powered tools including ChatGPT over traditional edtech platforms. "Publishers and other sites are feeling the pain from AI search. As they lose out on traffic, we're seeing publishers lean into subscriptions and paid AI licensing deals to bolster revenue," Emarketer analyst Minda Smiley said. GRAPHIC AI search ad spending is expected to constitute nearly 1% of total search ad spending this year and 13.6% by 2029 in the U.S., according to Emarketer. Sectors such as financial services, technology, telecom, and healthcare are embracing AI as they are seeing clear advantages in using the technology to enhance their ad strategies, while the retail industry's adoption is slow, the report said. Google recently announced the expansion of its AI-powered search capabilities into the consumer packaged goods sector through enhancements in Google Shopping.
The Guardian
an hour ago
- The Guardian
Abba singer Björn Ulvaeus teases fans in London with AI-assisted musical
After bringing a blockbuster hologram version of Abba to a purpose-built venue in east London, Björn Ulvaeus's next technological exploration is a musical that he's written with the help of artificial intelligence. Ulvaeus told an audience at SXSW London that he was 'three-quarters' of the way through writing a new musical which he has created with assistance from AI songwriting tools. He did admit the technology had limitations when it comes to songwriting, saying it was 'lousy at [writing a whole song]' and 'very bad at lyrics' but was helpful whenever the 80-year-old songwriter reached a creative impasse. 'You can prompt a lyric you have written about something, and you're stuck maybe, and you want this song to be in a certain style,' Ulvaeus said. 'You can ask it, how would you extend? Where would you go from here? It usually comes out with garbage, but sometimes there is something in it that gives you another idea.' The use of AI is a hugely contentious issue in the music industry. This year, Dua Lipa and Paul McCartney were among hundreds of artists who signed an open letter urging the prime minister to protect artists' copyright and not 'give our work away' to big tech. Ulvaeus said AI isn't the creative threat but rather a collaborator. 'It's fantastic. It is such a great tool,' he added. 'It is like having another songwriter in the room with a huge reference frame. It is really an extension of your mind. You have access to things that you didn't think of before.' The musical is far from Ulvaeus's first flirtation with technology. Abba Voyage, the concert which launched three years ago in a purpose-built arena that featured digital avatars of the band, has been a huge success and was described as a 'dazzling retro-futurist extravaganza' by the Guardian. It was a risky venture as Pophouse Entertainment, the company behind the project needed to bring in £140m in order to break even. He told the audience at SXSW London that he embraced experimental approaches to songwriting and studio technology much earlier in his career. This experimental approach is characteristic of Ulvaeus's career-long fascination with technological innovation. During his time in Abba, he along with bandmate Benny Andersson 'always wanted the latest thing', and were among the first to use the Mini Moog synthesizer and use digital recording machines in their studio. 'We were always on the lookout for, you know, you would hear a sound on the record. How was that done? And then you get that stuff,' said. The Abba songwriter told the audience that he sees AI as the latest evolution in his career rather than a threat which is how it is perceived by many. 'I actually wake up curious every morning,' Ulvaeus said, explaining the drive that has kept him creating decades after Abba's peak. 'Everything's really after our wanting to try new things.'
