Latest news with #Kaggle
Indian Express
21-04-2025
- Business
- Indian Express
Why the tech industry is obsessed with Chatbot Arena, the AI-ranking platform
With tech companies like OpenAI, Google, and Meta launching new AI models within weeks of each other, it's becoming increasingly difficult not only to keep track but also to assess how sophisticated each model truly is. That's where Chatbot Arena comes in—a free, crowdsourced benchmarking platform that tests newly launched AI models and pits them against each other across various parameters. In fact, ever since it was launched, Chatbot Arena has become the silicon valley's new obsession. Here's everything you need to know about Chatbot Arena—how it works and why the crowdsourced ranking site has become so popular. What is Chatbot Arena? Most companies measure their AI models against a set of general capability benchmarks, but there is no industry-standard benchmark or universally accepted method for assessing large language models (LLMs). Founded in 2023 by researchers affiliated to UC Berkeley's Sky Computing Lab, Chatbot Arena has emerged as the most practical—and virtually the only—tool to determine which AI model is the best on the market. Essentially, it's an interactive platform where users can pit multiple AI chatbots against each other in real-time conversations. Ranking of various AI models on the chatbot arena. What sets Chatbot Arena apart is that it allows AI models to converse freely across a wide range of topics, offering a more holistic assessment of their conversational skills. This is important criteria – after all, even small differences to factors like prompts, datasets, and formatting can have a huge impact on how a model performs. In Chatbot Arena, users can interact with the chatbots, get side-by-side comparisons of various AI tools with complete weakness and strengths, and vote on which one performs better. The tool becomes a testing ground for AI developers, researchers, and anyone who is interested in benchmarking these AI tools. Chatbot Arena recently transitioned into a full fledged company called LMArena, operating under Arena Intelligence Inc. The new company is co-founded by Dimitris Angelopoulos, Wei-Lin Chiang—another former UC Berkeley postdoctoral researcher—and Ion Stoica, a professor and tech entrepreneur. Chatbot Arena is funded through a combination of grants and donations, including support from Google 's Kaggle data science platform, Andreessen Horowitz, and Together AI. How does Chatbot Arena works? Perhaps the biggest reason why the AI benchmarking tool is so popular in the first place is that it makes it easy to compare two AI models side by side. Not only does Chatbot Arena allow users to pit the latest AI chatbots from OpenAI, Google, Anthropic, and Meta against each other, but its scoreboard also ranks over 100 AI models (developed by organisations or individuals) based on nearly 1.5 million votes. These rankings span a wide range of categories, including coding, long-form queries, mathematics, 'hard prompts,' and various languages such as English, French, Chinese, Japanese, Spanish, among others. It all adds up to why the AI benchmarking tool is so popular among the global community users. The industry also praises Chatbot Arena for offering neutral benchmarking, making the platform largely free of bias—an important factor for objective comparisons between different AI models. Chatbot Arena has partnerships with OpenAI, Google, and Anthropic to make their flagship models available for the community to evaluate. How to use Chatbot Arena Chatbot Arena offers two ways of evaluate different AI models, and if you are keen to try the AI benchmarking tool, make sure to try their 'battle' modes. The first mode is the Arena Battle, where your prompts is answered by Model A and Model B However, you don't know the model names until you click a button at the bottom. Then the model names appear. It's completely anonymous. To use Chatbot Arena battle: *Click OK on the popup that indicates this is a research preview. *Make sure to read the Terms of use to better understand how the battle works, then scroll down to the field that reads Enter text and press Enter. *Enter your prompt. *Click the Send button. Read the results and click the appropriate button. Now you should now see the name of the LLMs used for the battle. Another way to evaluate two AI models is through a side-by-side comparison. This allows you to choose the AI models of your choice and see how they perform against each other. It's definitely a better approach—at the very least, it helps you identify which model best suits your needs To use Chatbot Arena (side-by-side comparison): *Open in your web browser. *Click OK to the research preview popup. *Now click the top tab labeled 'Arena (side-by-side)'. *Click in the field showing the model name. *Select your model name from the drop-down list. You can also clear the field and start typing letters. *Scroll down and enter your prompt. *Click Send. *Review your responses and then Cast your vote using the buttons at the bottom. In case, you don't like the responses, you always click the Regenerate button.
Yahoo
17-04-2025
- Business
- Yahoo
AI benchmarking platform Chatbot Arena forms a new company
Chatbot Arena, the crowdsourced benchmarking project major AI labs rely on to test and market their AI models, is forming a company called Arena Intelligence Inc., reports Bloomberg. In a blog post published Thursday, Chatbot Arena said that the company will "give [it] the resources to improve [its platform] significantly over what it is today." The team also pledged to continue to provide neutral testing grounds for AI not influenced by outside interests. Founded in 2023, Chatbot Arena has become something of an AI industry obsession. Primarily run by UC Berkeley-affiliated researchers, Chatbot Arena has partnered with companies including OpenAI, Google, and Anthropic to make flagship models available for its community to evaluate. Chatbot Arena was previously funded through a combination of grants and donations, including from Google's Kaggle data science platform, Andreessen Horowitz, and Together AI. The organization's fledgling company hasn't disclosed any potential new backers yet — nor has it decided on a business model. This article originally appeared on TechCrunch at Sign in to access your portfolio
Yahoo
27-03-2025
- Automotive
- Yahoo
Mitsubishi Electric Researcher Named Kaggle Competitions Master in World's Largest AI Competition
Title, presented to Kaggle's top 1%, recognizes mathematical-optimization and machine-learning skills TOKYO, March 27, 2025--(BUSINESS WIRE)--Mitsubishi Electric Corporation (TOKYO: 6503) announced today that Dr. Kenji Ueda, a researcher at the company's Advanced Technology R&D Center in Japan, has been awarded the title of Kaggle Competitions Master by Kaggle, INC., a Google LLC subsidiary and the world's largest AI competition platform. Kaggle medals recognize competitors who use superior skills in mathematical optimization or machine learning to develop effective solutions to challenges posed by companies and government agencies. The title of Kaggle Competitions Master is awarded based on the total number of medals a competitor accumulates. Dr. Ueda was awarded his title as a result of accumulating one gold medal and two silver or higher medals in competitions over the past few years, thereby fulfilling the requirements for the award. Dr. Ueda has used his advanced skills and experience in mathematical optimization and machine learning, honed through daily research and development, to compete in Kaggle competitions in collaboration with internal and external team members. Previously, Dr. Ueda won a silver medal by placing 29th out of 1,054 teams in the Santa 2023 - The Polytope Permutation Puzzle competition to efficiently solve 3D puzzles using AI. He also won a gold medal by placing 10th out of 1,427 teams in the ARC Prize 2024 competition to harness the abstract reasoning abilities of AI. In addition, he secured a silver medal by placing 28th out of 1,514 teams in the Santa 2024 - The Perplexity Permutation Puzzle competition to minimize the perplexity of the large language model (LLM) by using AI to reorder English word sequences. For the full text, please visit: View source version on Contacts Customer Inquiries Advanced Technology R&D CenterMitsubishi Electric CorporationFax: + Media Inquiries Takeyoshi KomatsuPublic Relations DivisionMitsubishi Electric CorporationTel: +
