The AI Productivity Mirage: Costs Soar, Output Crawls
The widening gap between skyrocketing AI token costs and modest productivity gains refers to the growing mismatch between how fast enterprises’ AI infrastructure bills are increasing and how slowly their measurable output, efficiency, and task completion improvements are rising in return, despite aggressive adoption of advanced language models and coding assistants.
If AI is supposed to be the productivity engine of the decade, the current numbers look closer to a warning light than a success story. Venture capitalist Chamath Palihapitiya says his startup’s token costs are “doubling every forty-five days,” while productivity is up “maybe 5% max”. His own CTO’s explanation is damning: the easy wins from giving engineers AI tools are already banked, and each extra gain now needs far more tokens because “we’ve effectively already asymptoted”. At the same time, business leaders are speaking up about ballooning AI costs with thin returns. This is not an outlier; it is a pattern that exposes a flawed approach to AI cost calculation and a growing mismatch between hype and AI productivity ROI.
Token-Based Pricing Is Broken Without Task Completion
The industry’s default AI cost calculation—count tokens, multiply by price—fails where it matters: task completion. Databricks built an internal coding benchmark using real engineering tasks to study price versus performance for various models. Their conclusion is blunt: “Cheaper per-token does not imply cheaper per-task”. Anthropic’s Sonnet 5 was around 1.7 times cheaper than Opus 4.8 on a per-token basis, yet ended up more expensive per task: USD 2.09 (approx. RM9.60) for Sonnet 5 versus USD 1.94 (approx. RM8.90) for Opus. It also completed tasks less often—81 percent versus 87 percent—and burned more tokens along the way.
Price-per-token hides this reality and makes models with “expensive” tokens look worse than they are, while “cheap” models turn into silent budget drains when they fail more often or require repeated prompts. Databricks’ response was to build Omnigent, a wrapper to combine and swap multiple coding agents based on performance. That is the logical next step: AI cost models must be tied to price-per-task and completion rates, not to raw token counts or headline discounts. Until enterprises adopt that lens, token cost scaling will continue to outpace returns.
Open-Source AI: Four Months Behind, Ten Times Cheaper
While closed frontier models dominate headlines, open-source AI models are quietly reshaping the cost side of the equation. Databricks found that open-weight models like Z.ai’s GLM 5.2 score in the top capability tier and are statistically tied with Anthropic’s Opus 4.8 on quality, with GLM 5.2 costing USD 1.28 (approx. RM5.90) per task versus Opus’s USD 1.94 (approx. RM8.90). Another report argues that open-source models are roughly ten times cheaper per token than proprietary models.
Boris Renski says the fear narrative around frontier labs distracts from benchmarks that show open-source models are “only four months behind at a fraction of the cost”. Jonathan Bryce calls paying ten times more for a four-month lead “not an enterprise AI strategy,” but “a very expensive form of lock-in”. Featherless claims it can “slash frontier AI costs” through native optimization of GLM 5.2, and estimates that running 100 billion tokens monthly with GPT-5.5 costs USD 1,557,600 (approx. RM7,180,000) per year, while Claude Opus 4.8 costs USD 1,506,000 (approx. RM6,940,000). By contrast, their private cloud GLM 5.2 option charges a fixed annual rate of USD 90,000 (approx. RM415,000), saving over USD 1.46 million (approx. RM6,740,000) per year for a fully utilized team. For any cost-conscious AI strategy, ignoring open-source AI models at this point borders on negligence.
Enterprise AI Spending: From Token Legends to Reckonings
The spending trajectory is already colliding with reality. Uber’s CTO admits the company burned through its entire annual budget for tools like Claude Code and Cursor in four months. Meta had its own reckoning when an internal “Claudeonomics” leaderboard that ranked employees by token consumption and awarded titles like “Token Legend” leaked and was taken offline shortly after. These are not fun anecdotes; they are signs of AI experimentation turning into uncontrolled enterprise AI spending.
Palihapitiya warns that “everybody in the next three or four years will for sure go through” a similar reckoning. Meanwhile, open-source advocates argue that enterprises will look back at multi-year LLM contracts with frontier labs the way they now view legacy database licenses—overpriced, sticky, and hard to escape. The pattern is familiar: proprietary stacks win early, then open alternatives catch up and expose how much of the bill is for “commodity enterprise features” like connectors and observability, not intelligence itself. The message is clear: AI ROI cannot be justified by experimentation badges and internal memes. It must be grounded in disciplined AI cost calculation and real AI productivity ROI.
What a Sane AI Cost Strategy Looks Like Now
Enterprises are at a fork in the road. One path is to keep chasing leaderboard winners and accept token costs doubling every forty-five days while productivity sits at 5 percent or less. The other is to treat AI as infrastructure that must be tested, benchmarked, and constrained with the same rigor as any other capital project. That means measuring price-per-task, tracking completion rates, throttling “Ralph Wiggum loops” of repeated prompting, and building systems that can swap models as open and closed ecosystems evolve.
The numbers are already telling a story: open weight models like GLM 5.2 can match frontier performance at far lower per-task costs, open-source options are roughly ten times cheaper per token, and private cloud deployments can shave over USD 1.46 million (approx. RM6,740,000) off annual inference costs for heavy users. If AI is the new electricity, current spending patterns look more like leaving every light on than building a reliable grid. The next wave of winners will be the teams that treat AI not as a blank check, but as a disciplined, switchable utility.
