The AI Value Gap: Soaring Costs, Single-Digit Gains
The growing gap between AI infrastructure costs and software productivity ROI describes a pattern where token spending and AI-related software bills rise steeply while measurable efficiency gains inside organizations increase only modestly, forcing technology leaders to question whether current AI pricing models and usage patterns are financially sustainable.
That value gap is no longer an abstract concern; it now shows up clearly in real budgets. Business leaders say their AI costs are ballooning while promised productivity gains fail to materialize. Venture capitalist Chamath Palihapitiya describes his own software startup 8090 as a cautionary example: “Right now, our token costs are doubling every forty‑five days,” his CTO told him, while the productivity upside is “maybe 5% max”. When costs compound at that pace and output hardly moves, the math breaks.
The uncomfortable truth is that early AI wins are behind us. After the first wave of coding assistants and chatbots, each incremental improvement demands far more tokens and infrastructure spend, yet delivers smaller efficiency gains. If you manage an enterprise AI budget today, you are no longer paying for transformation—you are paying for marginal tweaks on an already asymptoting curve.
How Infrastructure Bills Are Being Pushed Onto Customers
AI infrastructure costs are enormous and rising fast. One consultancy estimates that the build cost for AI datacenters could reach USD 2 trillion (approx. RM9.2 trillion) by 2030. Those bills must land somewhere, and vendors have picked their answer: you. A major research firm warns that customers should brace for bigger software bills as software and AI vendors raise prices and add usage charges to pass their AI costs to customers.
That shift is already visible in AI pricing models. In the last six months, leading AI providers have moved key services away from flat-rate subscriptions toward usage-based billing, triggering cost concerns among users. As usage-based pricing models become more common, many organizations still lack the ability to forecast, monitor, and manage AI spending effectively. The result: bills that surprise finance teams and erode trust in AI programs.
Inside engineering-heavy companies, the impact is immediate. Uber’s CTO has said the company burned through its entire annual budget for tools like Claude Code and Cursor in four months as adoption jumped from 32% to 84% of its roughly 5,000 engineers, with individual engineers incurring monthly costs of USD 500–2,000 (approx. RM2,300–9,200) each. When AI usage feels like an open bar, vendors collect, and your margins pay for their datacenters.
Token Cost Calculation Is Broken: Task Completion Rules
The industry’s obsession with per‑token pricing is misleading. Cheaper tokens do not guarantee cheaper AI. Databricks built an internal coding benchmark using real engineering tasks to study price versus performance across models. Its conclusion cuts through the noise: what matters is cost per completed task, not the sticker price per token.
Their data shows why. One open‑weight model, GLM 5.2, matched a frontier model, Opus 4.8, in quality while costing USD 1.28 (approx. RM5.90) per task against USD 1.94 (approx. RM8.90) for Opus 4.8. But another model, Sonnet 5, looked cheaper per token yet turned out more expensive per task: USD 2.09 (approx. RM9.60) versus USD 1.94 (approx. RM8.90). The reason? It completed tasks less often—81% compared with 87%—and consumed more tokens in the process.
This is the heart of AI ROI: a model that fails more often forces humans back into the loop to redo work, multiplying both token costs and labor. If your AI infrastructure costs are rising while task completion rates stay flat, your software productivity ROI will degrade no matter how attractive your per‑token rate looks on paper.
Why Productivity Gains Are Flat and Layoff Narratives Ring Hollow
Enterprise teams are starting to say out loud what many suspected: AI adoption is not delivering the sweeping efficiency benefits they were sold. Leaders report that costs are ballooning but productivity is not. At 8090, the explanation is simple and sobering: the easy gains from giving engineers an AI coding assistant are already captured, and every additional unit of improvement requires disproportionately more tokens.
Meanwhile, the promise that AI would cut headcount rings hollow. A major research firm notes that staffing accounted for 35% of IT budgets in 2025, and 67% of tech decision‑makers expect to increase staffing budgets by 2027 while only 10% expect them to decline. It explicitly warns leaders to guard against inflated promises that AI can replace employees across the board.
What does fall, instead, is financial discipline. Without clear guardrails, teams fall into what Palihapitiya calls “Ralph Wiggum loops”—repeatedly firing the same prompt at a model until it stumbles onto an answer, accumulating large token bills with little to show beyond slow, expensive trial and error. That is not transformation; it is a productivity mirage funded out of your AI budget.
How to Rethink Your Enterprise AI Budget Before the Reckoning
The AI cost reckoning is coming whether boards are ready or not. Palihapitiya predicts that “everybody in the next three or four years will for sure go through it” as investors and executives confront the mismatch between spend and impact. He even argues that major AI vendors should rush to go public before that sentiment fully spreads. For enterprise buyers, the lesson is the opposite: slow down, measure, and demand real returns.
Research firms argue that the winners will not be the organizations that spend the most on AI, but those that invest in trusted data, strong governance, organizational readiness, and the ability to adapt as technology and customer behavior evolve. They also recommend adapting FinOps practices specifically to manage the unpredictable costs tied to AI usage. In practice, that means building internal benchmarks like Databricks did, tracking cost per completed task, and tying AI funding to measurable business outcomes, not demo‑driven enthusiasm.
AI infrastructure costs and AI pricing models are no longer a background detail—they are strategic risk. If token costs in your organization are doubling every 45 days while productivity is up 5% at best, the real question is no longer whether you can afford AI. It is whether you can afford to keep treating AI as magic instead of doing the hard work of proving software productivity ROI one task at a time.
