Token Costs Are Now the Real AI Bottleneck
Token pruning and model routing are AI model optimization techniques that reduce the number of input and output tokens processed by large language models, cutting enterprise LLM expenses by aggressively stripping unnecessary words, trimming prompts, and sending only the right tasks to the right models while maintaining task accuracy and usefulness.
The harsh reality is that enterprises are not slowing AI adoption because tools disappoint, but because the invoice does. One major ride-hailing platform reportedly burned through its entire annual AI budget in four months, while a global retailer introduced usage caps as a brake on runaway enterprise LLM expenses. As more tools move to per-token billing, every verbose answer, agent loop, and bloated context window turns into a line item. That pressure is so intense that companies are teaching models to talk like cavemen, proving that the problem is not the algorithms’ intelligence, but their verbosity and our careless use of tokens.
Caveman: Grassroots Prompt Compression, Industrial-Scale Signal
The caveman plugin is a grassroots prompt compression technique that attacks token pruning costs where they are most visible: the model’s mouth. A lightweight configuration file strips pleasantries, hedges, and transitions while preserving every line of code, turning friendly assistants into terse tools. Developer Julius Brussee built caveman after watching token spend disappear into chatbot politeness inside agent loops. His idea is blunt but effective: same substance, fewer words.
In controlled tests, Brussee reports a 65–75% reduction in output tokens compared with default verbose answers. Elastic Labs measured a 63.6% average reduction across eight Elasticsearch scenarios with no accuracy loss, and another walkthrough saw about 45% output savings and roughly 39% cost reduction. The tool already works with Claude Code, Codex, Gemini, and more than 30 coding agents, installed via one command. In practice, analysts estimate that code prose is only a slice of total usage, so real session savings may be closer to 4–5%—still meaningful, but not a full strategy.
When “Please” Becomes an Infrastructure Problem
Caveman’s popularity is a symptom of a deeper disease: enterprises mismanage tokens at every layer of their AI stacks. Senior leaders have publicly noted that users typing “please” and “thank you” into LLMs collectively cost their company tens of millions in electricity, and consulting findings link much of the soaring token spend to seemingly trivial uses like converting PDFs into presentations. This is what happens when frontier models meet ungoverned usage: polite language quietly becomes an infrastructure line item.
The response inside enterprises is getting sharper. One electrical and data center infrastructure firm circulated an internal memo calling out caveman as one of four high-impact cost practices, alongside avoiding powerful models and high reasoning settings by default. That document shows an important shift: instead of blindly chasing the latest model, companies now treat AI like any other shared utility, with explicit budgets, rationing, and optimization plans. The message is clear: prompt compression techniques are no longer a hack; they are policy.
Token Pruning and Model Routing: The Real Savings Engine
Despite the appeal of caveman-style outputs, critics are right that cutting response fluff only attacks a visible fraction of the bill. The heavier costs live in long input contexts, bloated prompt histories, and agent loops burning tokens in the background. Real AI model optimization comes from token pruning and smart routing: trimming what you send in, not only what you get out, and assigning tasks to the smallest adequate model instead of defaulting to a frontier giant.
Structural techniques are emerging as the playbook: prompt pruning to remove stale or redundant context, retrieval-augmented generation that injects only relevant data instead of entire databases, small-model routing for basic intake tasks, and token caching at around 10% of normal input prices. Combined with tools like caveman that can shrink output tokens by up to 75% in some coding scenarios, these approaches can systematically cut enterprise LLM expenses without sacrificing capability. The real competitive advantage will go to teams that treat tokens as scarce, not infinite.
Industry Wake-Up Call: From Frontier Fetish to Token Economics
The most telling sign that token pruning costs are now a board-level issue is who is contributing to the fixes. A director of engineering at OpenAI has contributed Codex plugin support straight into the caveman repository, and developers at OpenAI, Nvidia, and GitHub are reportedly using the tool. When senior insiders spend their time on cost-reduction plugins instead of only new model capabilities, the industry’s priorities are clear.
Looking ahead, we should expect formal AI style guides that set token budgets per workflow and even a new role: the “token economist”. Enterprises now face a strategic choice: keep chasing frontier models with unpredictable, high token costs, or adopt optimized routing strategies that pair smaller models, prompt pruning, RAG, and caching with slimmed-down outputs. The smart bet is obvious. AI will not be won by the company with the biggest model, but by the one that treats every token like money—because at enterprise scale, it is.
