Arthur Open-Sources First Real-Time AI Evaluation Engine

Associated Press

31-03-2025

Build. Experiment. Scale. Now With Open-Source AI Evaluation.
NEW YORK, March 31, 2025 /PRNewswire/ -- AI is evolving fast—but making it work at scale remains a challenge. Today, Arthur is launching the Arthur Engine, the first open-source, real-time AI evaluation engine designed to help teams monitor, debug, and improve Generative AI and traditional ML models. No black-box monitoring. No third-party dependencies. No data privacy risks. All for free.
Why Real-Time AI Evaluation Matters in 2025
As AI adoption grows, so do its risks. Without real-time evaluation, organizations face:
Data leaks— 8.5% of employee prompts contain sensitive data (Harmonic Security).
Model degradation— AI models drift over time without ongoing monitoring.
Debugging nightmares – Slow iteration cycles lead to poor model performance.
The Arthur Engine solves these challenges by providing instant visibility, real-time guardrails, and on-the-fly model optimization—right inside your own environment.
'AI is moving fast, and we need to ensure it moves in the right direction. Open-sourcing the Arthur Engine puts powerful AI evaluation tools into the hands of developers, researchers, and builders worldwide.'
— Ashley Nader, Lead AI PM at Arthur
What Makes Arthur Engine Different?
Unlike traditional AI monitoring tools, Arthur Engine runs locally—preserving data sovereignty and eliminating compliance risks.
Real-Time AI Evaluation – Instantly detect failures before they impact production.
Active Guardrails – Intervene in real-time to prevent hallucinations and bad outputs.
Customizable Metrics – Tailor evaluations to your specific AI use case.
Privacy-Preserving & Secure – Keep all data inside your infrastructure.
Works Across All Models – Supports GPT, Claude, Gemini, open weights models, and traditional ML.
'By open-sourcing Arthur Engine, we're making AI trust and safety accessible to all developers—allowing them to safeguard AI systems with fully customizable, high-performance monitoring tools.'
— Cherie Xu, Technical Lead, Machine Learning at Arthur
AI Evaluation, Built for the Future
The Arthur Engine is part of Arthur's broader AI performance monitoring suite, designed to help organizations:
Validate AI outputs in real time
Detect performance shifts before they become problems
Ensure regulatory compliance and explainability
This open-source release marks a new standard in AI transparency, security, and performance monitoring.
AI is reshaping the world—let's make sure it performs the way it should.
About Arthur
Arthur is the leading AI performance company, empowering organizations to monitor, measure, and improve machine learning and generative AI models at scale. Designed for trust, accuracy, and efficiency, Arthur helps organizations optimize AI performance with real-time insights, proactive model monitoring, and cutting-edge guardrails.
Backed by a research-led approach, Arthur delivers exclusive capabilities that enable teams to build, deploy and scale AI with confidence.
Founded in 2019, Arthur has raised over $60M in venture funding from Index Ventures, Acrew Capital, Greycroft, Work-Bench, and other top investors.