The New Epistemology Of Project Portfolio Reporting

Forbes

3 days ago

Distinguished Analyst and Research Fellow, Info-Tech Research Group. Insights and recommendations for projects, portfolios and change.
Project portfolio reporting is losing its foundation in science as prediction and measurement are replaced by sentiment, abstraction and AI. Every executive should be prepared to justify which projects deserve funding, cancellation or continuation.
Portfolio decisions were historically based on business plans and return on investment forecasts in an imperfect but scientific iterative cycle. We articulated what we intended, predicted the costs and outcomes, executed the plan, measured the results, incorporated the insights and repeated the cycle for the next project. It was inherently scientific, as long as we kept learning from the gap between intention and outcome.
But a lot of us didn't.
This science is brittle, easily shattered when execution begins without a plan. The old planning paradigm ends when organizations stop predicting project costs and resource requirements.
With the coarsening of advanced planning, dissolution of resource forecasting, decay of executive commitment and obfuscation of outcomes, portfolio reporting is increasingly losing its foundation in that iterative science. Epistemology, the theory of knowledge or 'how we know what we know,' is morphing into something new.
Three Alternative Scientific Approaches
Three portfolio approaches have emerged that still have scientific foundations.
• Capacity And Demand: Estimate your resource capacity, match it up with the most urgent demand and keep people focused on what matters now. This approach taps into urgency as the ultimate truth and focuses on today. It ignores tomorrow's science for today's firefighting, but it is fundamentally scientific.
• KPIs And Scorecards: This approach leans on key metrics for alignment, performance, value and tracking. Truth comes from measuring observations against targets. This approach is scientific, though its own validity is derived from the underlying math.
• Governance Feedback: Review results, measure benefits realization and govern with reviews and audits. This approach supports decision making with strategies, outcomes and authoritative action. It maintains a scientific foundation through prediction, measurement and authoritative direction.
These scientific approaches to portfolio reporting continue to produce usable, accessible insights for organizations focused on the underlying cycle of predict-execute-measure.
Modern Abstractions
Two newer approaches are grounded in more abstract models:
• Enterprise Architecture (EA): This leverages roadmaps focused on capability and value representing business, data and technology dimensions. This is an academically elegant model based on high-maturity analysis by mostly senior, credentialed resources. This approach requires completeness of the analysis, and the abstraction comes when justification for future spending is inferred from the capability map.
• Agile/Product Management: These are roadmaps that convey strategic vision and future initiatives through the lens of a product or product family and its backlogs. This approach favors a strong, informed product view and promotes rapid decision making by highly engaged product teams. The risk comes from having a strong focus on product without sufficient balance across the portfolio when product demand outpaces resource supply.
There's a lot to like in these newer, abstracted approaches. For EA, we get the benefit of our most senior and credentialed thinkers creating a robust academic model to guide our view. For product management, we get the benefit of roadmaps produced by product specialists with deep expertise and lots of skin in the game.
Both of these approaches offer high-resolution viewpoints, but can become self-referential if they're not measuring anything—the capability map or roadmap devolves into a justification exercise, rather than a forecasting discipline.
The Rise Of Storytelling
One of the fastest growing portfolio reporting concepts I've seen in the industry is based on storytelling. We see vivid narratives that promote clear outcomes with minimal risk, but based on minimal supporting evidence. This approach may have sound, defensible research supporting it, but the reader can't be sure if it's substantiated. The quality of the story renders the science opaque, so decision makers are mostly responding to sentiment, even when the science is solid.
Storytelling has improved the user experience, but it often replaces clear insight with artful rhetoric, making the outcome engaging but unverifiable.
AI Has Entered The Building
Generative AI is set to overtake human-authored storytelling with elegant narratives that appear to be structured and scientific. The most exciting value comes when we feed the AI model with solid forecast and performance data: I may or may not be the better writer, but my AI tool can produce something in seconds that takes me days or weeks.
The downside risk is that epistemology becomes opaque when we blend facts, inferred trends, creative assumptions and colorful rhetoric into business cases and progress updates. The burden of evidence shifts to the reader to separate the steak from the sizzle.
Rebuilding Your Portfolio Epistemology
The next time your portfolio reporting gets scrutinized, re-examine the concept of 'how you know what you know" by considering the following questions:
• Are your reports based on prediction and measurement?
• Are they driven by financial and resource scarcity?
• Are the insights aligned with accepted measures and KPI's?
• Do they require a logical abstraction to decode the spending justification?
• Are the reports painting compelling pictures without supportive evidence?
• Have you adopted an AI narrative without critically evaluating the robustness of evidence?
At the end of the day, decision makers are trying to understand whether 'value' represents a defensible measure of economic value or an unexamined narrative. That distinction between measurement and narrative is what defines your portfolio epistemology. Re-establishing it requires an objective assessment of the following question: 'How do we know this, and how is it proven?'
As we enter the era of AI storytelling and systemic elegance, epistemic clarity is your best defense.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?