What 1,000 Tokens Per Second Really Means for GPU Inference Speed
High GPU inference speed describes how quickly a model can generate or prefill tokens on hardware that is widely available, and it is fast becoming a core metric for judging model optimization, inference cost reduction, and the practicality of running advanced AI on standard GPUs instead of specialized accelerators. Xiaomi’s MiMo-V2.5-Pro UltraSpeed mode is a clear example: developed with TileRT, this 1-trillion-parameter model surpasses 1,000 tokens per second while still running on general-purpose GPUs. Earlier, MiMo-V2-Flash topped out at 150 tokens per second, already fast enough that text came out quicker than most people can read. UltraSpeed mode is roughly ten times faster than standard MiMo-V2.5-Pro API access, trading higher throughput for a higher API rate. As Xiaomi limits access to an application-based trial, the release signals that ultra-fast inference is no longer confined to premium enterprise hardware.
Xiaomi MiMo-V2.5-Pro UltraSpeed: Speed Gains and Cost Trade-offs
MiMo-V2.5-Pro’s UltraSpeed mode shows how carefully tuned model and system co-design can push commodity GPUs to enterprise-grade throughput. Xiaomi describes UltraSpeed as a “3x price increase” that delivers a “10x output experience” compared with the standard MiMo-V2.5-Pro API. The base model charges 0.025 yuan per million tokens on a cache hit, 3 yuan on a cache miss for input, and 6 yuan per million tokens for output, making the new tier a premium option focused on throughput-sensitive workloads. For developers, the question is whether higher tokens per second offset the higher rate through lower wall-clock time and better GPU utilization. In chatbots, live agents, and high-volume content generation, faster decoding can shorten queues and cut per-request overhead. Xiaomi’s limited trial, with capped sessions and queue entries, hints at how scarce high-speed inference resources still are, even as they begin to reach standard GPU fleets.
MiniMax M3 and Sparse Attention: Long Context at Lower Overhead
Where Xiaomi chases raw decoding speed, MiniMax M3 focuses on smarter computation for long-context tasks. Running on NVIDIA’s accelerated infrastructure, this 428-billion-parameter multimodal model supports text, images, and video with a context window of up to one million tokens. Its defining feature is MiniMax Sparse Attention, which reduces computational overhead and supports faster prefill and decoding compared with the previous generation. By skipping needless attention operations, M3 can maintain high GPU inference speed even as context length grows, which is vital for extended coding sessions, video analysis, and complex design workflows. The model was trained natively on multimodal data instead of adding capabilities later, so it can handle large, mixed workloads without awkward pipelines. With deployment paths through TensorRT LLM, SGLang, and vLLM, M3 makes high-throughput, long-context reasoning feasible on standard NVIDIA hardware, rather than niche, bespoke accelerators.
From Enterprise Accelerators to Commodity GPUs: A New Cost Curve
These launches point to a broader shift: instead of assuming only top-tier accelerators can deliver acceptable tokens per second, vendors are pushing commodity GPUs much harder. Xiaomi proves a trillion-parameter model can exceed 1,000 tokens per second through tight model–system co-design, while MiniMax uses sparse attention and expert routing to deliver long-context performance without proportional cost growth. For startups and mid-market teams, this opens new choices. Rather than renting specialized hardware, they can prioritize model optimization and inference cost reduction on standard NVIDIA platforms. Throughput directly shapes economics: faster generation can mean fewer GPUs, shorter runtimes, and more predictable latency for real-time agents, customer support bots, and continuous content workflows. As more models reach high throughput on general-purpose GPUs, the buying decision shifts from “which accelerator?” to “which model architecture and runtime give the best cost per useful token?”.
