The Broken Promise: When AI Math Stops Making Sense
The widening gap between AI spending and productivity ROI is the growing imbalance between rapidly compounding token costs and the flattening efficiency gains that real businesses are seeing once early automation wins have already been captured.
The core story is ugly: one software startup reports its AI token costs are doubling roughly every 45 days while productivity gains are stuck at around 5 percent. That is not innovation; it is negative compounding. It matches what many leaders now admit privately: AI invoices keep rising while output barely moves. In parallel, fitness operators are pouring money into AI with almost no measurable return, despite industry-wide enthusiasm. On the technical side, even data platforms that benchmark models are concluding that cheap tokens often cost more per completed task, undermining headline pricing. Put together, the numbers point to a single conclusion: AI cost efficiency has collapsed, not because models lack power, but because the economic model around them is mispriced and poorly implemented.
Tokens Are Compounding, Productivity Is Not
Chamath Palihapitiya’s startup 8090 is the clearest illustration of today’s token cost analysis problem: his CTO told him that “our token costs are doubling every forty-five days” while the productivity gain is “maybe 5% max.” In other words, the marginal return on each additional token is collapsing. Early gains from AI coding assistants have been “asymptoted,” so every extra sliver of improvement requires far more tokens and prompts.
The waste is often self-inflicted. Engineers fall into so-called “Ralph Wiggum loops” — firing the same or similar prompts repeatedly until the model stumbles onto something useful, racking up bills with little incremental value. At scale, this becomes a budget problem, not a curiosity. One large ride-hailing platform burned its entire annual budget for AI coding tools in four months as usage among roughly 5,000 engineers surged and individual bills hit USD 500–2,000 (approx. RM2,300–RM9,200) a month. Those numbers reveal the real productivity ROI gap: compounding spend with almost flat output.
Most AI Projects Fail for Human Reasons, Not Technical Ones
If AI technology is so capable, why is the productivity ROI gap so wide? In the fitness sector, where AI is “dominating conversations,” very few brands see a measurable return. Karl Foster, head of AI at a gym software company, calls it an “uncomfortable truth” that AI fails to produce measurable ROI 95 percent of the time. Yet he insists the underlying technology has proven value; the problem is everything around it.
The failures are mainly organizational. Data is fragmented across memberships, marketing, payments, and communication systems, and at least 70 percent of projects report data integration as the primary barrier. Content that models rely on, such as FAQs and websites, is often outdated, degrading AI quality over time. Culture is worse: staff fear replacement and resist tools they see as threats, unless leadership frames AI as a way to augment staff and improve output two to three times rather than cut headcount. Without executive commitment and realistic timelines, projects stall in “pilot purgatory” long before the 18–36 months it typically takes for real value to appear.
Price per Task, Not per Token: The Only Cost Metric That Matters
The industry’s obsession with per-token price is a distraction from AI cost efficiency. A data software firm recently built an internal benchmark of real coding tasks and compared multiple models on price and performance. Its finding: models with more expensive tokens could be cheaper overall because they complete more tasks, with fewer retries, at higher quality. One open-weight model matched a leading frontier model on capability while costing $1.28 per task against $1.94 for the frontier system.
Another pair of models shows why price-per-token is misleading. One mid-tier model was around 1.7 times cheaper per token than a premium option, but ended up costing more per task — $2.09 versus $1.94 — because it completed only 81 percent of tasks compared with 87 percent and burned more tokens in the process. As academics have shown, in about a third of comparisons, the model with the lower listed price ended up more expensive overall. The conclusion is blunt: true AI cost calculation must center on task completion rates and business outcomes, not headline token prices.
From Hype to Discipline: Closing the Productivity ROI Gap
Across fitness and enterprise software, the pattern is the same: high enterprise AI spending, scant productivity gains, and growing economic inefficiency. Leaders push AI into every corner of the business, then act surprised when fragmented data, culture shock, and undisciplined usage drain budgets. Dashboards gamifying token burn — like internal leaderboards that label heavy users “Token Legend” — only amplify the problem before they are quietly shut down.
A reset is overdue. On the vendor side, many will be forced to move away from selling raw tokens and toward higher-value services that are accountable to clear business metrics. On the buyer side, the path forward is narrow but clear: start with a single, painful business problem, define success in task completion and revenue or cost impact, measure AI cost efficiency per task, and require leadership-level commitment. Anything else is another step into a negative-sum game where token costs compound faster than any plausible gain. The broken promise can be fixed, but only if the industry stops worshipping AI usage and starts paying for AI outcomes.
