Why Eval++ Is the Next Great Compute Primitive — Sunil Pai & Matt Carey, Cloudflare

Cloudflare's infrastructure primitives—particularly Durable Objects, Workers, and Dynamic Workers—enable building stateful, production-ready AI agents with b...

By Sean Weldon

Infrastructure Primitives for Stateful AI Agents: An Analysis of Cloudflare's Distributed Computing Architecture

Abstract

This paper examines Cloudflare's infrastructure primitives—Durable Objects, Workers, and Dynamic Workers—as foundational technologies for building production-ready, stateful AI agents at planetary scale. The analysis investigates how these primitives address critical challenges in serverless computing: state persistence, real-time synchronization, and secure execution of dynamically generated code. Key findings include achieving 15ms latency through edge network distribution, enabling resumable streaming without database replication, and providing secure sandboxing for arbitrary code execution through capability-based isolation. The integration of Model Context Protocol (MCP) servers demonstrates practical applications supporting concurrent agent instances at scale. These architectural innovations eliminate traditional distributed systems complexity while enabling novel use cases in generative UI, multiplayer AI collaboration, and safe execution of user-generated code. The findings suggest that fundamental infrastructure design decisions can resolve entire classes of distributed systems problems previously requiring extensive engineering effort.

1. Introduction

The deployment of AI agents in production environments has exposed fundamental architectural limitations in traditional serverless computing platforms. Conventional serverless architectures operate statelessly, treating each function invocation as an isolated, ephemeral execution context. While this model simplifies horizontal scaling and resource allocation, it necessitates external state management through databases or caching layers for applications requiring persistent context. For AI agents that must maintain conversational history, execute long-running background tasks, and coordinate across multiple user sessions, this architectural constraint introduces latency penalties, operational complexity, and scalability bottlenecks.

Furthermore, the emergence of AI-generated code and dynamic user interfaces presents novel security challenges. Current industry practice relies on intermediate representations—typically JSON schemas describing UI components—to avoid executing untrusted code directly. This approach, while secure, constrains the expressiveness of AI-generated interfaces and introduces unnecessary serialization overhead. The conventional prohibition against using eval() or similar dynamic code execution primitives reflects legitimate security concerns but limits architectural possibilities for AI systems that generate executable code.

Durable Objects represent a paradigm shift in serverless architecture by maintaining persistent class instances addressable by unique identifiers, where all requests and connections for a given ID route to the same instance. Combined with Dynamic Workers—isolated execution environments for arbitrary code—these primitives provide the foundation for stateful AI agents that can execute background tasks, maintain persistent connections, and safely run generated code without traditional infrastructure overhead. This analysis examines how these infrastructure primitives enable production-ready AI agents, focusing on technical architecture, performance characteristics, security models, and practical applications in real-world systems.

2. Background and Related Work

2.1 The Statelessness Constraint in Serverless Systems

Traditional serverless platforms, exemplified by AWS Lambda and similar services, treat compute instances as ephemeral and interchangeable. This design simplifies infrastructure management but creates challenges for applications requiring state persistence. AI agents, which must maintain conversational context, tool execution history, and scheduled task queues, typically require multiple database round-trips per request to reconstruct state. This architecture introduces latency penalties and necessitates complex coordination logic for features such as real-time synchronization across multiple client connections.

2.2 Model Context Protocol and Stateful Connections

The Model Context Protocol (MCP), which gained adoption in early 2024, defines a standard for stateful, bidirectional connections between AI clients and capability servers. MCP's architectural requirements—persistent connections, session state, and real-time message exchange—conflict fundamentally with stateless serverless models. Production implementations of MCP servers have historically required traditional cloud infrastructure with load balancers, sticky sessions, and database-backed state replication. Cloudflare's implementation of MCP servers on Durable Objects, including integrations with PayPal, Sentry, Linear, and Intercom, represents one of the first production deployments that eliminates this infrastructure complexity by leveraging stateful primitives natively.

2.3 Code Generation Security Constraints

Current approaches to AI-generated user interfaces rely on intermediate representations to avoid executing untrusted code. The "Jason Bender pattern" and similar architectures generate JSON schemas that client-side renderers interpret, rather than generating executable code directly. This constraint emerged from legitimate security concerns: executing arbitrary code generated by language models or users presents risks of data exfiltration, resource exhaustion, and unauthorized access. The thirty-year prohibition against using eval() in production code reflects these concerns. However, this constraint limits the expressiveness of AI-generated interfaces and introduces unnecessary complexity in serialization and deserialization logic.

3. Core Analysis

3.1 Durable Objects: Stateful Serverless Architecture

Durable Objects solve the statelessness problem by maintaining a persistent class instance per unique identifier. Unlike traditional serverless functions that spawn new instances for each request, Durable Objects ensure that all requests and connections associated with a specific ID route to the same instance. This architectural decision provides several critical capabilities for AI agents.

First, Durable Objects enable sub-frame latency for interactive applications. The implementation achieves 15ms latency in London—just under the 16ms threshold required for 60 FPS rendering—by leveraging Cloudflare's planetary network topology. This performance characteristic results from strategic hardware placement near Internet Service Providers rather than centralized data centers, enabling edge computation with minimal network traversal.

Second, Durable Objects support long-running background tasks, hibernation, and persistence without requiring external database state management. An AI agent implemented as a Durable Object can maintain conversational context, execute scheduled tasks (such as "every Friday at X time, compile Git history and wiki, send to manager"), and maintain connections to external services—all within a single addressable compute unit. The TLDraw application demonstrates this capability through real-time collaborative synchronization across multiple devices, achieving perfect state consistency without custom distributed systems engineering.

Third, the architecture scales to millions or trillions of concurrent instances. Each Durable Object operates independently, eliminating coordination overhead between instances. This design enables applications to create dedicated agent instances per user, conversation, or workflow without resource contention or coordination complexity.

3.2 Agents SDK: Production-Ready AI Infrastructure

The Cloudflare Agents SDK extends Vercel's AI SDK with production features specifically designed for stateful agents. The SDK provides first-class support for tool calls, cross-tab synchronization, and resumability—capabilities that typically require custom distributed systems engineering.

Resumable streaming represents a particularly significant technical achievement. When a user refreshes their browser mid-stream during an AI response, the agent automatically reconnects and continues from the beginning without requiring database replication or sticky sessions. This capability emerges naturally from the Durable Objects architecture: because all connections for a given agent ID route to the same instance, the instance maintains complete streaming state and can replay or continue responses for reconnecting clients.

The SDK exposes React hooks and plain JavaScript clients for defining callable functions and background scheduling. Developers can implement agent capabilities as simple JavaScript functions without managing message queues, task schedulers, or state synchronization logic. This abstraction eliminates the need for developers to become "crazy distributed systems engineers" to achieve production-grade reliability and performance.

3.3 Dynamic Workers: Secure Arbitrary Code Execution

Dynamic Workers enable the execution of arbitrary code strings—from users, customers, or language models—in isolated worker instances without pre-deployment. This capability challenges the conventional prohibition against eval() by providing what the presenters term "eval plus plus": a fast, secure, and economically viable alternative to traditional code execution models.

The security model begins with a zero-capability sandbox. Dynamic Workers start with no access to fetch(), no API access, and no environment variables exposed. Capabilities are explicitly granted from outside the sandbox through an allowlist model. For example, a Dynamic Worker might receive permission to "only allow outgoing fetches to github.com/xyz" while remaining isolated from all other network resources. This capability-based security model inverts traditional approaches, which grant broad permissions by default and attempt to restrict dangerous operations.

The implementation leverages V8 isolates rather than full virtual machines or containers. Isolates provide lightweight isolation without complete file systems or VM overhead, enabling the platform to spin up billions of instances on demand. This architectural decision makes Dynamic Workers economically viable for use cases that generate and execute code at scale, such as AI agents that produce custom business logic or users who submit code for execution.

3.4 Generative UI and Multiplayer AI Applications

The combination of Durable Objects and Dynamic Workers enables novel architectural patterns for AI-generated interfaces. Rather than generating JSON schemas for client-side interpretation, agents can generate React components or HTML directly and execute them safely in Dynamic Workers. This approach eliminates the serialization intermediary while maintaining security through sandbox isolation.

Cloud Artifacts demonstrates client-side rendering of untrusted HTML, but server-side rendering of untrusted code requires stronger security guarantees. Dynamic Workers provide these guarantees, enabling agents to generate and execute code in the cloud before rendering results in user interfaces. This architectural pattern extends beyond UI generation to general-purpose code execution, enabling use cases that previously required domain-specific languages or JSON-based configuration.

The infrastructure also enables multiplayer AI experiences. Current AI chat platforms, exemplified by ChatGPT, do not support shared conversation links or multi-user collaboration. Durable Objects eliminate the distributed systems engineering required to achieve real-time synchronization, streaming resumability, and multi-tab/multi-browser coordination. These capabilities emerge automatically from the architecture: because all connections route to the same Durable Object instance, state synchronization across clients requires no additional coordination logic.

4. Technical Insights

4.1 Performance and Latency Characteristics

The 15ms latency achieved in London results from Cloudflare's network topology rather than software optimization alone. Hardware placement near ISPs, combined with bulk bandwidth agreements and cross-border byte agreements, enables edge computation with minimal network traversal. This structural advantage—resulting from business model decisions made a decade prior—provides sustainable performance characteristics independent of computational optimizations.

4.2 State Management and Storage Integration

Durable Objects support SQLite storage directly within instances, enabling structured data persistence without external database dependencies. Larger files can be stored in R2 (Cloudflare's object storage), while @cloudflare/shell provides a complete virtual file system abstraction. This storage hierarchy enables developers to select appropriate persistence mechanisms based on data characteristics and access patterns.

4.3 Code Bundling and Dependency Resolution

The Worker Bundler pulls dependencies from NPM, strips TypeScript types and JSX syntax, and generates executable code. Critically, the bundler uses Cloudflare's cache for dependency availability rather than relying directly on NPM infrastructure. This design improves reliability and reduces latency for dependency resolution during deployment.

Language support extends beyond JavaScript: Python is a first-class language, while other languages compile to WebAssembly. Zig produces particularly small WASM bundles compared to Go and Rust equivalents, making it suitable for resource-constrained edge deployment.

4.4 Economic and Accessibility Considerations

Cloudflare's pricing model ($5/month standard tier with free tier available) reflects structural cost advantages rather than loss-leader marketing. The infrastructure economics enable multi-million dollar SaaS businesses to operate on free accounts, demonstrating that edge computing architectures can achieve fundamentally different cost structures than centralized cloud platforms.

5. Discussion

The infrastructure primitives examined in this analysis suggest that fundamental architectural decisions—network topology, isolation mechanisms, and state management models—can eliminate entire classes of distributed systems problems. Traditional approaches to building stateful AI agents require developers to implement custom solutions for state synchronization, connection management, and task scheduling. Durable Objects internalize these concerns within the infrastructure layer, enabling developers to focus on business logic rather than distributed systems engineering.

The integration of MCP servers on Durable Objects demonstrates that emerging protocols requiring stateful connections can be deployed on serverless infrastructure when appropriate primitives exist. This finding challenges the assumption that stateful protocols necessitate traditional server deployments with dedicated infrastructure. As AI systems increasingly require persistent connections for tool use, streaming responses, and multi-turn interactions, infrastructure primitives that natively support these patterns become essential.

The capability-based security model in Dynamic Workers represents a significant departure from traditional sandboxing approaches. By starting with zero capabilities and explicitly granting permissions, the model inverts the security paradigm from "deny dangerous operations" to "allow only specified operations." This approach may prove more robust for AI-generated code, where the space of potentially dangerous operations is difficult to enumerate exhaustively.

However, several questions remain unaddressed. The scalability limits of individual Durable Object instances are not specified; applications requiring coordination across millions of concurrent connections to a single agent may encounter bottlenecks. The economic viability of the free tier for large-scale applications raises questions about long-term sustainability and potential pricing changes. Finally, the vendor-specific nature of these primitives creates migration challenges for applications built on this infrastructure.

6. Conclusion

This analysis demonstrates that Cloudflare's infrastructure primitives—Durable Objects, Workers, and Dynamic Workers—provide foundational technologies for building production-ready, stateful AI agents at scale. The architecture achieves sub-frame latency through edge network topology, enables resumable streaming without database replication, and provides secure sandboxing for arbitrary code execution through capability-based isolation. These capabilities emerge from fundamental design decisions in network placement, isolation mechanisms, and state management rather than incremental optimizations.

The practical implications extend beyond AI agents to any application requiring stateful serverless computation, real-time synchronization, or safe execution of user-generated code. The integration of MCP servers, the Agents SDK, and Dynamic Workers demonstrates that these primitives enable previously impractical use cases while eliminating traditional distributed systems complexity. As AI systems increasingly require persistent state, long-running operations, and safe code execution, infrastructure primitives that natively support these patterns will become critical for production deployments. Future work should investigate scalability limits, economic sustainability, and portability considerations for applications built on these architectural foundations.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub