The Illusion of Speed: Why Anecdotes About AI Productivity Fall Short
Engineering leaders increasingly hear that AI coding tools are making developers dramatically faster, yet the underlying numbers rarely align. GitHub cites 55% faster task completion with Copilot in controlled experiments, while Gartner projects 25–30% productivity gains by 2028 for organisations that apply AI across the full development lifecycle, and only 10% for today’s code-generation tools. A separate 2024 Gartner survey found that just 34% of teams using generative AI report high productivity gains. Meanwhile, DORA’s 2025 research links AI adoption to higher throughput but also lower delivery stability, and other studies even show experienced engineers taking longer with large language models. These figures are often quoted as if they describe the same outcome, yet they differ in scope, time horizon, and measurement basis. Without consistent engineering metrics AI, leaders are left budgeting against marketing claims and anecdotes, not evidence from their own delivery pipelines.
From DORA Metrics to AI ROI: Turning Delivery Data into Business Insight
Google Cloud’s DORA framework offers a structured way to convert software delivery performance into business value, making it central to credible AI ROI measurement. The latest DORA report on AI-assisted development frames AI as an amplifier: it magnifies existing strengths or dysfunctions rather than fixing broken systems. The model starts with foundational capabilities such as a quality internal platform, disciplined version control, and AI-accessible internal data. Improvements in these areas then surface as better DORA metrics—throughput, lead time, change failure rate, and time to restore. These engineering signals flow into non-financial outcomes like developer experience and customer satisfaction, and finally into financial results such as cost savings and revenue growth. The report explicitly warns that its sample ROI calculations are high-uncertainty estimates, intended to prompt local modelling. For leaders, the message is clear: AI ROI must be derived from their own DORA framework metrics and cost structures, not generic industry averages.
The J-Curve and the Hidden Costs of AI-Assisted Development
One of DORA’s most important insights for engineering metrics AI is the J-Curve of value realisation. Most organisations will see a temporary dip in productivity when they introduce AI into their development cycle tracking. This downturn stems from three main factors: a learning curve as teams reshape workflows around AI, a verification tax as engineers review AI-generated code, and pressure on downstream processes—testing, change approvals, deployment—caused by increased code volume. DORA’s model treats this period as the “tuition cost of transformation,” not as a sign that AI is failing. The research also highlights an instability tax: AI usage correlates with higher individual effectiveness but also with greater software delivery instability, as fragile pipelines struggle under faster change. Rather than slowing adoption, the framework argues for investing in automated testing, continuous integration, and small, frequent releases so the eventual gains outweigh the temporary disruption.
Why Strong Engineering Foundations Are Prerequisite for AI Success
Both DORA’s research and independent platforms like Navigara emphasise that AI’s value depends on the strength of existing engineering foundations. DORA’s findings show that AI amplifies high-performing organisations while exacerbating legacy bottlenecks in struggling ones. Where internal platforms are brittle, workflows unclear, or deployment processes heavily manual, AI tends to create local productivity spikes that disappear in downstream chaos. Navigara’s introduction of Engineering Throughput Value (ETV) underscores this point. ETV is a per-commit metric that scores each team’s work against its own pre-AI baseline, making it possible to see whether claimed productivity gains actually show up in the codebase and delivery pipeline. This commit-level view also reveals when AI-driven throughput is eroding stability. For engineering leaders, the implication is non-negotiable: before expecting AI ROI, organisations must invest in robust platforms, disciplined version control, and reliable automation that can safely absorb faster change.
What Engineering Leaders Should Really Track to Justify AI Investments
To move beyond hype, leaders need concrete engineering metrics AI that connect AI usage to measurable outcomes. First, anchor AI ROI measurement in DORA framework metrics: deployment frequency, lead time for changes, change failure rate, and time to restore service. Track these before and after AI adoption, and segment by team or service to avoid masking local regressions. Second, incorporate new measures like Engineering Throughput Value to examine commit-level impact against a pre-AI baseline, rather than relying on task-level speed anecdotes. Third, explicitly model verification tax and instability tax as costs in your ROI calculations, alongside tooling and training investments. Finally, tie engineering improvements to business KPIs such as incident impact, feature cycle time, and customer-facing quality. As the DORA team puts it, the question is not how much code AI writes, but which bottlenecks it actually clears—and whether the data supports that story.