16-06-2025
Why Product Analytics And Experimentation Must Converge
Dan Rogers, CEO of LaunchDarkly.
getty
For too long, software teams have been forced to choose between knowing what's happening and understanding why. Product analytics told us where users dropped off, but not what would have worked better. Experimentation lets us test new features, but often with little context about where to start or which users to target. That separation has created blind spots, bottlenecks and bad decisions.
Here's the reality: Observing user behavior without testing hypotheses is passive. Testing ideas without grounding them in real data is reckless. And in today's fast-moving software economy, where AI is reshaping everything from feature behavior to user expectations, neither approach on its own is enough.
That's why the smartest companies aren't just experimenting—they're converging experimentation with product analytics to form a single, continuous learning loop.
Product analytics and experimentation were never meant to operate in isolation. Yet in many companies, they still do.
Analytics teams study dashboards and funnel reports, trying to extract insights weeks after a release. Meanwhile, product and engineering teams run A/B tests that aren't always informed by behavioral data or worse, aren't measured rigorously post-launch. It's a disconnected process that leads to slow iteration, guesswork and features that underperform.
This siloed model might have worked a decade ago. It doesn't anymore.
In today's environment, where user expectations shift rapidly and AI models behave unpredictably, the only way to build confidently is to create a real-time loop between insight and action.
When analytics and experimentation converge, every behavior pattern becomes a hypothesis to test. Every test becomes a data point to analyze. Every decision becomes more grounded, targeted and measurable.
Take a familiar example. Let's say your analytics show users abandoning the checkout flow at the payment stage. Without experimentation, you might guess it's the form layout, rewrite some code and hope conversion improves. But when you unify analytics and experimentation, you can design an experiment with different form layouts, deliver those layouts to specific user segments (like first-time buyers versus returning customers) and track conversion alongside other behavioral signals. In a matter of days, you're not just identifying what's broken—you're discovering how to fix it, who it affects most and what the downstream impact will be.
Savage X Fenty (a client of LaunchDarkly) offers an example of how some companies are integrating experimentation into their day-to-day operations. By embedding testing directly into workflows, they've been able to move more quickly and identify useful insights earlier in the process.
This same model is proving critical in AI-powered products, which are inherently unpredictable. With traditional development, teams can test deterministic logic. But with AI, you're managing variables like prompt structure, model drift and real-time learning. Unified experimentation and analytics allow AI teams to iterate on models and parameters in real time while monitoring performance, user satisfaction and potential risks.
It starts with identifying where users struggle. Instead of guessing, teams can use analytics to reveal friction points like areas of drop-off, hesitation or confusion. From there, they form hypotheses rooted in actual behavior, not hunches. Experiments are then crafted to target those behaviors, often delivered to different user segments to see how responses vary. Once experiments are live, results flow directly into the same analytics infrastructure that tracks overall product usage, ensuring teams aren't evaluating changes in a vacuum.
Over time, this becomes a habit. Teams observe, test, learn and refine. Not once, but continuously.
Unifying product analytics and experimentation isn't just a more efficient way to work—it fundamentally changes the way teams build. Product managers, engineers and data scientists begin to operate from a shared reality. Instead of siloed reports and speculative ideas, they have a common, evolving source of truth.
This is how modern software development should function. Continuous delivery needs continuous learning. Anything less is leaving value and velocity on the table.
The companies that get this right won't just build faster. They'll build smarter. They'll ship products that are tuned to their users, backed by evidence and constantly improving. They'll foster a culture of curiosity, rigor and resilience. And in a world of constant change, that mindset becomes the true competitive advantage.
Because today, the winning teams aren't just the ones who move quickly—they're the ones who learn even faster.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
