Session-Based Compute: The New Unit of Cloud Work
Session-based compute is an emerging cloud execution model where platforms allocate and manage long-lived, stateful environments per session instead of routing each independent request to any available backend worker, so that AI agents can keep context, identity, and security guarantees tightly bound over many steps.
The key shift is that AI agent runtimes are no longer treating calls as disposable requests; they are treating conversations and workflows as first-class sessions. Over the past few months, AWS, Microsoft, Google, and Anthropic each shipped agent runtime updates that point to the same architectural shift. At its core, the agent runtime is becoming a control plane for state, identity, isolation, and lifecycle, and the unit it schedules has moved from the individual request to the agent session. This is not a cosmetic change to load balancers—it is a redefinition of what the cloud considers a unit of work. The convergence across four platforms suggests session-aware execution is becoming a baseline requirement for production-grade agents.
Why Traditional Microservices Broke Under Stateful AI Execution
The old cloud stack assumed that any backend could serve any request and that state lived elsewhere. That model worked for web APIs and microservices because requests did not depend on one another and state could sit in Redis or a database. Agents destroy both assumptions. Enterprise agents are not ordinary API calls; they are long-running, stateful, tool-using processes that often run code influenced by user input.
Agents pose two distinct challenges: preserving conversational state across turns and enforcing a security boundary strong enough to withstand untrusted code. Routing alone can keep a conversation on the same replica, but that does nothing for isolation when the agent executes model-generated code shaped by user input. A shared kernel does not give untrusted, agent-generated code the tenant isolation that enterprise security teams require. The better mental model is no longer a traditional load balancer; it is closer to a virtual actor runtime, where an addressable identity is instantiated on demand, kept active while needed, and deactivated when idle, with one live instance per key.
Four Providers, One Session Model, Many Isolation Strategies
Over the past few months, four AI giants quietly rebuilt the same thing at once: AWS, Microsoft, Google, and Anthropic each shipped agent runtime updates that point to the same architectural shift. They converge on session-aware execution but diverge sharply on how they isolate those sessions. AWS AgentCore is the most opinionated: each session gets a dedicated Firecracker microVM with isolated compute, memory, and filesystem, and a request carrying the same runtimeSessionId is routed back to that microVM through a session header; when the session ends, the microVM is terminated and its memory sanitized.
Microsoft rebuilt its Foundry-hosted agents in April around per-session isolation. The platform creates a per-session VM-isolated sandbox on demand, runs it, and tears it down at session end, with no replica count and no warm pool to size; each agent gets a dedicated Microsoft Entra identity, and sessions run a 15-minute idle timeout against a 30-day maximum lifetime. Google’s Agent Engine keeps request scaling inside the reasoning loop, with minimum and maximum instance counts and a container_concurrency that defaults to 9, while pushing untrusted code into an isolated Code Execution sandbox and externalizing state into Sessions and a Memory Bank. Anthropic’s Managed Agents virtualize the agent into a session that logs everything, a near-stateless harness that runs the loop and routes tool calls, and a sandbox where code runs.
Security, Isolation, and the Asana Warning Shot
This architectural shift is not an academic exercise; it is a response to concrete failures of cloud infrastructure isolation. An agent can execute model-generated code shaped by user input, so the backend must be treated as a security boundary rather than a generic compute target. When that boundary fails, the impact is immediate for ordinary users. When Asana disclosed a flaw in its MCP server in June 2025, the server had been live since May 1, a window of about five weeks; a tenant-isolation check failed, and around 1,000 organizations could see project data belonging to other customers.
That incident showed how brittle the old model is when user identity, agent context, and session state are not consistently bound together. According to the report, data crossed organizational boundaries even without an external attacker involved. For an enterprise buyer, the key question is who owns that binding and how it is tested under concurrent, multi-tenant load. Session affinity and clever routing are not enough. Platforms must tie lifecycle, isolation, and identity to the session itself, and they must prove that binding under pressure, or risk repeating Asana’s mistake at much larger scale as agents move into critical business workflows.
What Session-Aware AI Infrastructure Means for the Next Wave
Once correctness and security depend on binding identity, state, and isolation, the session becomes the unit that the platform schedules. AWS documents that an AgentCore session remains Active while it processes work and sits Idle when it is provisioned but waiting, and it reaches Terminated after a 15-minute idle timeout or an 8-hour maximum lifetime. The agents hosted on Microsoft Foundry follow a different pattern, with the same 15-minute idle timeout but a 30-day maximum lifetime. Those numbers encode opinionated bets about how long AI agents should live, how much state they should keep, and what kind of workloads they target.
In summary, the important shift is not that cloud providers have found a better load balancer for agents; it is that the agent session is becoming the operational unit of enterprise AI infrastructure. Many enterprise platforms will end up combining more than one of these patterns. No single architecture is the right answer for every enterprise workload: a multi-day research agent fits Azure’s longer-lived session, a code-heavy agent needs microVM-grade isolation, and high-volume automation favors a lighter isolate. The next wave of AI agent runtimes will be judged less on model benchmarks and more on how reliably they handle stateful AI execution, cloud infrastructure isolation, and cost for thousands of concurrent sessions. The winners will be the platforms that treat session-based compute as a first-class operating system for agents rather than an add-on to legacy microservices thinking.
