Sessions, Not Requests, Are Becoming the New Unit of Compute
Session-based compute for AI agent runtimes is an architectural model where cloud platforms treat each long‑running, stateful agent conversation as the primary unit of scheduling, isolation, and billing, replacing traditional request-centric load balancing with session-aware execution that binds identity, state, and security into a single operational context. Over the past few months, AWS, Microsoft, Google, and Anthropic have all rebuilt their agent runtimes around this idea, signaling that the “session” has graduated from a web UX concept to a first‑class cloud primitive. This is not cosmetic plumbing work; it is a strategic admission that agents are qualitatively different from stateless APIs. Enterprise AI is shifting from quick model calls to autonomous workflows that think, call tools, run code, and wait. These behaviors break the old assumptions that any worker can serve any request and that state lives safely off to the side.
Why Agents Broke the Old Cloud Model
The traditional cloud infrastructure stack scaled on a simple premise: frontends like NGINX or HAProxy route each HTTP request to any available backend, while shared state sits in Redis or a database so replicas stay interchangeable. That model works as long as requests do not depend on one another and backends can remain stateless. AI agents violate both conditions. They are conversational, maintain evolving context across turns, and often execute code influenced by user input, which turns the runtime into a security boundary rather than a generic compute pool. Session affinity and consistent hashing have already made load balancers behave more like rudimentary session routers, but routing alone cannot guarantee per‑tenant isolation for untrusted code. The Asana MCP incident—where a failed tenant‑isolation check let about 1,000 organizations see project data from other customers—shows how dangerous it is when user identity, agent context, and session state are not tightly bound.
Four Cloud Providers, Four Session Isolation Strategies
Once isolation and correctness hinge on the binding between identity, agent context, and session state, the session becomes the unit the platform schedules. Here, the major providers agree on the destination but disagree on the route. AWS AgentCore is the most opinionated: every agent session gets its own Firecracker microVM with isolated compute, memory, and filesystem, and subsequent requests tagged with the same runtimeSessionId are routed back through a dedicated session header. Microsoft rebuilt its Foundry-hosted agents in April explicitly around per-session isolation, favoring longer‑lived environments suited to multi‑day workflows. Anthropic’s Managed Agents split the architecture into a logging session, a near‑stateless harness that runs the loop and routes tools, and an isolated sandbox where code executes. Google takes a hybrid approach: its Agent Engine keeps request scaling inside the reasoning loop with configurable instance counts and a default container_concurrency of 9, while moving untrusted code into a separate Code Execution sandbox and externalizing conversation state to Sessions and Memory Bank.
Developer Experience: AWS Blocks and the Rise of Session-Aware Backends
Session-aware execution is not staying buried in platform internals; it is shaping developer tools. AWS recently released AWS Blocks in public preview, an open-source TypeScript framework where each “Block” bundles application code, a local development implementation, and the AWS infrastructure needed for production. At preview, approximately 20 Blocks cover databases, authentication, AI agents and knowledge bases, file storage, real-time messaging, background jobs, scheduled tasks, and email, and there is no additional charge for Blocks itself—teams pay only for the underlying AWS services they use. The design premise is explicitly agent-driven: “AI agents write code, and the framework provides the correct way to write it from the start.” Developers can run npm run dev to get a local stack—with Postgres, auth, messaging, and file storage—without an AWS account, then deploy the same code to Lambda, DynamoDB, Aurora, API Gateway, and Bedrock with zero changes. Blocks applications are CDK applications, so teams can embed Blocks into existing cloud infrastructure stacks and evolve toward session-based, agent-centric backends without a full rewrite.
What Enterprises Must Decide Next
With AWS, Microsoft, Google, and Anthropic converging on the notion that the agent session is the operational unit of enterprise AI infrastructure, session-aware execution is becoming the production baseline rather than a niche optimization. The economics follow: AWS documents that an AgentCore session is Active while processing work, Idle when provisioned but waiting, and Terminated after 15 minutes of idle time or an 8‑hour maximum lifetime—these are lifecycle semantics for long‑running environments, not ephemeral HTTP requests. For an enterprise buyer, the question is not whether to adopt session-based compute but which session isolation strategies match their risk and performance profile. A code-heavy agent may warrant microVM-level isolation, while high‑volume automation might favor lighter isolates, and multi‑day research agents may rely on longer‑lived sessions. No single architecture wins outright, and many platforms will mix patterns. This is how cloud infrastructure evolves: existing primitives recombined around a new abstraction. The next generation of autonomous AI systems will be designed, billed, and secured at the session boundary.
