Redefining GPU inference speed on standard hardware
GPU inference speed refers to how many output tokens per second a model can generate when running on graphics processors, and it now depends as much on model optimization and system design as on raw hardware power, with new techniques such as UltraSpeed modes and sparse attention achieving breakthrough performance on standard GPUs that previously required premium accelerators. This shift matters because most developers run large models on consumer and mid-tier cards, where latency and throughput costs can stall real-world deployment. Recent launches from Xiaomi and MiniMax show that architectural tweaks and software-level acceleration can push large models beyond once-impossible token per second thresholds while still running on widely available infrastructure. Instead of waiting for the next expensive GPU generation, teams can now look at smarter scheduling, precision formats, and attention patterns to achieve faster inference and more responsive applications.
Xiaomi MiMo-V2.5-Pro UltraSpeed: 1,000+ tokens per second
Xiaomi’s MiMo-V2.5-Pro UltraSpeed mode is a clear example of software-led performance gains on general-purpose GPUs. Co-developed with TileRT, the 1-trillion-parameter model now breaks the 1,000 tokens per second barrier in generation, far beyond the 150 tokens per second reported for MiMo-V2-Flash at its launch in December 2025. Xiaomi attributes this leap to an “ultimate co-design” of the model and its underlying system, suggesting heavy work on kernel scheduling, caching, and parallelism rather than a dependence on exotic hardware. The company says UltraSpeed delivers “roughly 10 times faster output than standard MiMo-V2.5-Pro API access,” framing the mode as a premium performance tier. That tier is priced at a “3x price increase” over the standard MiMo-V2.5-Pro API and is currently limited to an application-based trial, with approvals prioritized for enterprises and professional developers and free, time-limited chat sessions as a testbed.
MiniMax M3 and sparse attention for long-context workloads
Where Xiaomi pushes decoding throughput, MiniMax M3 attacks a different bottleneck: long-context processing. Built with 428 billion parameters and a context window of up to one million tokens, M3 must handle massive histories without crippling compute costs. Its answer is MiniMax Sparse Attention, which selectively focuses computation across the context window to reduce overhead. According to TestingCatalog, this design “enabl[es] substantially faster prefill and decoding than its predecessor,” making one-million-token use cases more practical on NVIDIA hardware. The model trains natively on multimodal data—including text, images, and video—so it can handle complex pipelines like extended coding sessions, video analysis, or design workflows without separate adapters. Support for BF16 and MXFP8 precision formats, plus up to 128 experts per token, further aligns M3 with modern inference engines such as TensorRT LLM, SGLang, and vLLM on Blackwell-class GPUs.
Practical gains: from consumer GPUs to enterprise pipelines
These approaches converge on the same goal: making large models practical on non-exotic GPU infrastructure. Xiaomi’s UltraSpeed mode shows that 1,000+ token per second generation is possible on standard GPUs through careful co-design, not only on enterprise-grade accelerators. MiniMax M3’s sparse attention, long context, and expert routing aim to keep multimodal workloads responsive on NVIDIA platforms that enterprises already use. For developers, this means less waiting and lower latency for chatbots, coding assistants, and analysis tools, even when context windows stretch into hundreds of thousands of tokens. It also shifts optimization focus from model size alone toward throughput, context length, and energy efficiency. In effect, GPU inference speed becomes a product feature: something users can choose in tiers, benchmark in tokens per second, and align with different application needs from short-form interaction to marathon reasoning sessions.
Free trials and API competition to democratize fast inference
Performance alone is not enough; access matters. Xiaomi is running an application-based UltraSpeed trial from June 9 to June 23, 2026, granting approved users two weeks of free chat access with guardrails such as daily queue limits, 30-minute session caps, and automatic release of idle resources. On the MiniMax side, M3’s presence in NVIDIA’s API catalog, with a free endpoint highlighted by TestingCatalog, turns high-speed, long-context inference into a weekend project option rather than a budget line item. Developers can test large multimodal workflows, or connect always-on agents, without up-front commitments. These limited yet meaningful free paths signal a competitive push among vendors: high token per second rates and sparse attention tricks are not only marketing metrics but gateways to broader adoption, as more teams experience what fast, optimized GPU inference feels like in real applications.
