From Turn-Based Chats to Continuous Real-Time Multimodal AI
Thinking Machines is challenging the familiar stop-and-go rhythm of today’s chatbots with what it calls interaction models—real-time multimodal AI systems designed to collaborate continuously across audio, video, and text. Instead of waiting for a user to finish typing or speaking, these new audio video AI models listen, watch, and respond at the same time. The company’s flagship TML-Interaction-Small model reportedly answers in about 0.40 seconds, a conversational AI latency close to human reaction speeds. Demos highlight scenarios such as counting exercise reps from live video, tracking posture, and translating speech while sustaining natural dialogue. Under the hood, the system processes interaction in 200-millisecond slices, allowing it to stream understanding and responses rather than batch them. This marks a shift from command-response interfaces toward AI that behaves more like an attentive collaborator than a passive tool.
Inside the 0.4-Second Architecture: Micro-Turns and Background Reasoning
The technical premise behind Thinking Machines AI is that interactivity must scale alongside intelligence. To achieve sub-second responsiveness, the interaction models break time into micro-turns—200-millisecond chunks in which the system simultaneously listens, sees, and generates output. A time-aware interaction model manages the live conversation, handling cues like overlapping speech and changes in video scenes. In parallel, an asynchronous background model tackles heavier reasoning and tool use, then feeds results back into the ongoing dialogue without interrupting it. This dual-track design mimics how humans can talk while planning what to say next. Instead of the rigid request-then-response pattern common in today’s systems, these audio video AI models maintain a continuous, two-way channel. The result is lower perceived latency, fewer awkward pauses, and a more fluid conversational flow that feels closer to talking with another person than querying a machine.
Why Sub-Second Conversational AI Latency Matters for Human Interaction
Latency is more than a technical metric; it shapes whether an interaction feels natural or frustrating. Thinking Machines’ claim of 0.40-second response times for TML-Interaction-Small brings conversational AI latency into a range where users can interrupt, clarify, and riff in real time, as they would with another person. In practical terms, this means the AI can respond to a raised eyebrow on video, a mid-sentence correction in audio, or a new text input without resetting the conversation. Continuous, multimodal awareness also reduces the need for users to repeat themselves or over-specify instructions. The system can infer intent from tone, gestures, or visual context, not just words. Together, these features hint at interfaces where AI is less a separate “tool” and more a present participant—joining meetings, coaching workouts, or assisting creative work without breaking the flow of human conversation.
New Use Cases: From Collaborative Workflows to Ambient AI Presence
Real-time multimodal AI unlocks scenarios that traditional turn-based systems struggle with. In the demos, the AI counted exercise repetitions from video while chatting, noticed when someone slouched, and translated speech live—illustrating how continuous perception and response can support coaching, accessibility, and monitoring tasks simultaneously. In collaborative work, an interaction model could follow a video call, listen to side comments, and update documents or dashboards in the background as people talk. It could also act as a persistent, ambient presence in physical spaces, reacting to visual cues and conversation without explicit prompts. Because the models are built for native audio, video, and text, developers can design experiences where users switch channels seamlessly—speaking, typing, or showing something to the AI without changing modes. This blended interaction pattern could redefine productivity tools, customer support, education, and entertainment experiences.
Research Preview Today, New Expectations Tomorrow
For now, Thinking Machines’ interaction models remain a research preview, with access limited to selected partners and wider availability promised later this year. Yet even at this early stage, the approach is reshaping expectations for what real-time multimodal AI should deliver. Early benchmarks cited by the company suggest TML-Interaction-Small outperforms existing systems in both intelligence and interaction quality, reinforcing the idea that better collaboration is as crucial as bigger models. The startup’s leadership, including founder Mira Murati, is explicitly targeting the “bandwidth bottleneck” between humans and AI, arguing that how we work with these systems can no longer be an afterthought. As other providers race to lower response times and add richer modalities, interaction models like these may become the reference point: AI that doesn’t just answer questions, but participates in the ongoing fabric of human conversation.