Stop Giving Your Agent Tools. Give It a Sandbox with Mikyo King, Arize AI

Code Mode - giving coding agents a programming environment to search for and compose tool calls rather than a static catalog of tools - reduces context bloat

By Sean Weldon

Stop Giving Your Agent Tools. Give It a Sandbox

Abstract

Tool-calling agents conventionally receive a static catalog of callable functions, with every tool response accumulated into the context window across iterative turns. This architecture degrades as tool inventories, context requirements, and task horizons scale upward. This synthesis examines Code Mode, an approach in which an agent is given a tool-search capability and a programming environment rather than an enumerated tool menu, enabling it to sequence, parallelize, and conditionally compose tool invocations while returning only necessary outputs to the context window. Drawing on Arize AI's production implementation inside its Phoenix observability platform, evaluated using the Harbor sandbox framework, this analysis finds a 43% reduction in token consumption and a 31% reduction in cost with approximately constant latency. The paper also catalogs operational risks - sandbox lifecycle failures, secrets exposure, double sandboxing - and unresolved problems including durability, language-dependent determinism, and multimodal handoff, concluding with practical implications for agent architecture design.

1. Introduction

Software capability has historically bifurcated along a skill boundary: engineers operated with scriptable, composable primitives, while non-engineers were restricted to vendor-defined interfaces and purchased software. Coding agents dissolve this boundary, because software that previously had to be purchased or hand-coded can now be generated dynamically at the point of need.

Anything you need, the coding agent can actually build that software for you dynamically.

Foundation model laboratories are training explicitly toward this capability, as evidenced by the prominence of benchmarks such as SWE-bench. The architectural implication is that agents should increasingly be aligned to construct the software they require on demand, rather than to select from a fixed inventory of pre-built affordances presented as a tool catalog.

This raises a concrete design question: does replacing a static tool catalog with a programming environment - Code Mode - measurably improve agent efficiency and unlock behaviors unavailable to conventional tool-calling, and at what operational cost? Section 2 establishes the baseline tool-calling architecture and its failure modes. Section 3 defines Code Mode, explains the mechanisms underlying its advantages, and describes Arize's production implementation. Section 4 consolidates actionable technical findings. Section 5 discusses implications and open problems, and Section 6 concludes.

2. Background and Related Work

The traditional tool-calling agent architecture supplies a large tool catalog to the model upfront. The agent selects a tool, the runtime executes it, and the full result is appended to the context window; this loop repeats until task completion. The design is adequate for small, fixed tool sets but degrades under three pressures simultaneously: demand for more emergent agent behavior, larger context requirements, and longer task horizons.

Relevant prior concepts include progressive disclosure of tools, which defers tool definitions until they are actually needed, and composability patterns exemplified by Claude's dynamic workflows and sub-agent composition. The evaluation methodology discussed later draws on reinforcement-learning conventions, specifically binary (1/0) task scoring and pass@K reporting, operationalized through Arize's Harbor framework.

3. Core Analysis

3.1 Failure Modes of Traditional Tool-Calling

Three specific degradation mechanisms are observable in conventional tool-calling architectures. Context rot arises from irrelevant tool definitions occupying the window alongside verbose or low-signal tool responses. Pagination overhead occurs when retrieving a complete result set requires repeated calls, each of which consumes additional context. Turn inflation forces compaction, which in turn increases latency, cost, and time-to-value. A representative failure case is PII handling: without a dedicated stripping tool, a traditional agent risks leaking sensitive data such as credit card numbers directly into the context window, because no intermediate processing step exists outside of what the tool catalog explicitly provides.

3.2 Code Mode as an Architectural Alternative

Code Mode replaces the enumerated tool menu with two capabilities: the ability to search for specific tools, and a programming environment in which to execute them.

You don't give it just the ability to execute the tool, you actually give it a programming environment to execute those tools in different types of flows.

Four mechanisms explain why this improves outcomes. First, progressive disclosure means tool definitions enter context only when relevant, rather than all upfront. Second, composability allows sequencing, parallel fan-out, and conditional chaining of tool calls - multiple tools can be invoked in a single turn rather than one call per turn. Third, token efficiency results from the agent writing code that filters and transforms tool outputs before anything reaches the context window, so "it only has to put in exactly what's necessary into the context window to solve its problem." Fourth, emergent behavior follows from the agent's ability to construct missing capabilities on the fly - for instance, writing PII-stripping logic dynamically rather than requiring a dedicated, pre-built tool.

3.3 Production Implementation at Arize

Arize built its own agent harness inside Phoenix, its open-source observability platform, through incremental expansion of the sandbox environment. The implementation began with Bash, composing Linux primitives (sed, grep) with Arize's own CLI and offloading intermediate results to files rather than context. A Python sandbox was added next, into which REST CRUD endpoints for trace retrieval and dataset manipulation were hoisted directly as callable functions. SQL capability followed, using text-to-SQL to support aggregate queries - such as filtering by P95 latency - without paginating through entire trace sets. Finally, a UI-bound programming paradigm was built, allowing the agent to manipulate time ranges and read data directly from the front-end data store, running inside a web worker with two-way data binding for computer-use-style UI interaction.

4. Technical Insights

Several implementation-relevant findings emerge from Arize's experience:

5. Discussion

The Arize results suggest that the tool-calling paradigm's core assumption - that an agent benefits from seeing a complete, upfront catalog of discrete affordances - is not well-suited to tasks involving large or evolving tool surfaces. Replacing enumeration with search-plus-execution shifts the bottleneck from context capacity to sandbox reliability and code-generation quality. This is consistent with the broader industry trend toward coding agents as a general-purpose substrate: if a foundation model can reliably write correct code (as benchmarks like SWE-bench are designed to measure), then tool orchestration becomes a special case of code generation rather than a separate capability requiring bespoke scaffolding.

However, the open problems identified - durability/resumability, language-dependent determinism, multimodal serialization, and the capability gap between frontier and smaller/cheaper models - indicate that Code Mode is not yet a drop-in replacement for all agent architectures. In particular, the dependency on frontier-level code-generation capability means organizations using smaller or cheaper models may not realize the same efficiency gains, since the approach's benefits are contingent on the model's ability to write correct, composable code reliably without extensive debugging iterations.

A further implication concerns evaluation infrastructure. Arize's use of Harbor - disabled network access, multiple trials, binary verifier scoring, pass@K tracking - represents a methodological commitment to measuring agent efficiency and reliability as first-class metrics alongside task success. This suggests that organizations adopting Code Mode should expect to invest comparably in evaluation tooling as in the sandbox implementation itself, since the efficiency claims (43% token reduction, 31% cost reduction) are only credible given rigorous, repeated-trial measurement.

6. Conclusion

This analysis indicates that giving coding agents a programming environment rather than a static tool catalog - Code Mode - produces measurable efficiency gains without latency penalty, based on Arize's production deployment inside Phoenix and evaluation via Harbor. The mechanism driving these gains is a combination of progressive tool disclosure, composable execution, selective context inclusion, and the capacity for emergent, on-the-fly tool construction.

The practical takeaway for engineering teams building agentic systems is that sandbox-based execution should be evaluated as a viable alternative to tool-catalog expansion, particularly as tool inventories and task horizons grow. Teams considering this shift should budget for the associated operational complexity - resource-constrained sandboxes, secrets vaulting, debuggability trade-offs - and should pair any such migration with rigorous trial-based evaluation infrastructure to validate claimed efficiency gains before broad deployment.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub

Sean Weldon builds apps and custom software at 22Atlas.

Start your build

More about Sean