From Single-Lab Loyalty to Multi-Model AI Routing
Multi-model AI routing is an enterprise strategy where software systems dynamically direct each prompt or workload to different large language models through a central gateway, optimizing for price, performance, and security instead of relying on a single AI vendor for every task. That shift is no longer experimental; it is the core of how serious teams are cutting AI costs while scaling. Vercel’s CEO Guillermo Rauch is blunt that picking one lab and building everything on it is now obsolete. Coinbase CEO Brian Armstrong is making the same bet, wiring production infrastructure to route work across several models instead of a single provider. The takeaway is clear: loyalty to one frontier model has turned into a tax, and engineering leaders are refusing to pay it.
Why the Token Free-For-All Had to End
The first wave of enterprise AI was a spending binge: pick a favorite lab, wire everything to its flagship model, and encourage teams to burn tokens as fast as they could. That era collapsed under its own bill. Companies realised that more tokens did not equal more customer value, and that frontier models were being wasted on routine tasks. At the same time, capabilities converged. Frontier models now sit much closer together on everyday engineering work, while open-weight and alternative models have improved and the price gap keeps widening. When multiple models can handle most workloads, sticking with one provider stops being strategic and starts being negligent. Trying to pick the single best AI provider is a losing game.
Coinbase: Cut Spend in Half, Grow Usage
Coinbase is the clearest proof that multi-model AI routing is a cost strategy, not a science project. The company cut its internal AI spend by nearly half while overall token consumption continued to increase, and it did so without imposing usage caps on engineers. The architecture runs through an internal LLM gateway that defaults to lower-cost open-weight models like GLM 5.2 and Kimi 2.7. GLM 5.2 is priced around USD 1.40 (approx. RM6.44) per million input tokens and USD 4.40 (approx. RM20.24) per million output tokens, compared with Anthropic’s Opus 4.8 at about USD 5 (approx. RM23) for input and USD 25 (approx. RM115) for output. That is a three- to six-times reduction per token, yet GLM 5.2 still scores 62.1 on SWE-bench Pro, beating GPT-5.5 at 58.6. According to the reporting on Armstrong’s architecture, Coinbase now operates roughly 1,200 full-time AI agents in production.
Gateways, Smart Routing, and the New AI Control Plane
The technical and strategic heart of this shift is the LLM gateway. Instead of asking engineers to pick models by hand, the gateway acts as a control plane: it intercepts every prompt and decides, in real time, which model should handle it. That decision weighs task complexity, latency, cache state, and current pricing. Armstrong’s approach is pragmatic: use frontier models for complex planning, but route execution-heavy or straightforward tasks to cheaper models that perform equally well. Coinbase then locks conversations to the same model while cache remains valid, pushing cache hit rates from around 5% to 60% and turning reuse into a major cost driver. As foundation models become easy to swap in and out, the true engineering challenge shifts to the surrounding infrastructure, not the model itself.
Vendor Diversification as Standard Enterprise AI Practice
Rauch’s view is that every piece of the AI stack—from model to gateway—is now plug and play. You can route to OpenAI, Anthropic, Gemini, or Chinese models like DeepSeek and GLM-5.2, depending on the workload and budget. Gemini in particular is gaining adoption because of what Rauch calls "awesome price/performance characteristics" at scale. This is classic vendor diversification strategy: similar to how teams moved from choosing a single cloud provider to multi-cloud for resilience and cost optimisation. In practice, Vercel already routes more than a trillion tokens per day across millions of deployments through this kind of multi-model AI infrastructure. Multi-vendor AI strategies are becoming standard practice for serious enterprise AI scaling, and the winners will be those who treat the LLM gateway as a strategic asset, not an afterthought.
