A Split Stack for Real-Time Voice Applications
OpenAI has expanded its audio offering with three OpenAI voice models tailored for live, interactive systems: GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper. Rather than relying on a single, monolithic assistant, developers can now assign distinct models to reasoning, real-time translation, and live voice transcription workloads. All three are exposed through the OpenAI Realtime API, targeting use cases where applications must keep speaking, handle interruptions, and call tools without losing context. This split design is aimed squarely at voice app development teams building assistants, call flows, and tool-using agents. By separating tasks, they can route simple turns to lightweight models while reserving deeper model capacity for complex reasoning. The approach also promises clearer operational visibility: when a live system stumbles, engineers can test whether the failure lies in transcription, translation, or reasoning instead of treating the entire voice surface as a single black box.
GPT-Realtime-2: GPT-5-Class Reasoning for Live Voice
GPT-Realtime-2 is the reasoning core of the GPT-Realtime voice lineup, bringing what OpenAI describes as GPT-5-class reasoning to spoken interaction. It is built to manage longer, tool-heavy conversations while staying responsive as users talk, interrupt, and change direction. The model’s context window has been expanded from 32K to 128K tokens, helping it maintain state across long calls, tool hops, and follow-up questions. To make voice agents less brittle, GPT-Realtime-2 introduces conversational behaviors such as short spoken preambles like “let me check that,” clearer verbal cues when it calls tools, and explicit acknowledgements when a task fails instead of falling silent. Developers can tune reasoning levels from minimal to xhigh, trading latency against depth depending on the turn. In OpenAI’s internal benchmarks, higher settings significantly outperform GPT-Realtime-1.5 on Big Bench Audio and Audio MultiChallenge, positioning GPT-Realtime-2 as the tier for decision-heavy, context-rich voice interactions.
GPT-Realtime-Translate: Real-Time Translation for Multilingual Voice
GPT-Realtime-Translate focuses on live multilingual conversations, functioning as a dedicated real-time translation API within the broader stack. It can translate speech from over 70 input languages into 13 output languages while keeping pace with the speaker, and it can optionally surface realtime transcriptions alongside the translated audio. This makes it suitable for customer support, cross-border sales, education, and media scenarios where audiences expect immediate, localized experiences. Early adopters highlight how this lane can be embedded directly into existing voice surfaces. Deutsche Telekom is testing it for multilingual voice interactions, while Vimeo has showcased a demo that translates a product education video on the fly as it plays, without requiring a separate dubbed version. By decoupling translation from the main reasoning model, teams no longer need to bolt translation logic onto a general assistant, and can instead optimize latency, accuracy, and cost specifically for multilingual workloads.
GPT-Realtime-Whisper: Streaming Speech-to-Text for Live Transcription
GPT-Realtime-Whisper fills the transcription lane with a streaming speech-to-text system tuned for low latency. It delivers live voice transcription as people speak, supporting use cases such as live captions, meeting notes generated during a conversation, and workflows where spoken language must be processed instantly. Because it is separate from the reasoning tier, developers can keep transcription fast and efficient without pushing every utterance through the most computationally intensive model. This separation also aids reliability and debugging. If live voice transcription remains accurate while a complex reasoning task slows or fails, operators can focus tuning efforts on the reasoning layer instead of replacing the entire voice stack. That modular design is especially valuable for support desks, travel assistants, and media pipelines, where one lane can be optimized for speed and another for decision depth. Together with GPT-Realtime-2 and GPT-Realtime-Translate, GPT-Realtime-Whisper completes a pipeline that captures, translates, and understands speech in real time.