One Day, $1,000 in Tokens: An AI War Room for Legacy Code - Ford Prior, GEICO

GEICO accelerated its slow 'outer loop' (collaboration, decision-making, and governance around major software changes) by building a multiplayer AI-native 'w...

One Day, $1,000 in Tokens: An AI War Room for Legacy Code

Abstract

This synthesis examines GEICO's approach to accelerating the outer loop of software delivery - collaboration, decision-making, and governance - through a multiplayer AI-native war room built on fleets of Claude-based agents. While coding agents have compressed the inner loop of software development, governance and stakeholder synthesis remained the binding constraint, particularly within a regulated insurance enterprise where risk management is embedded throughout the software delivery lifecycle. The methodology involved three bounded experiments - uplift, transform, and reimagine - each targeting agent-callable API delivery into legacy claims systems spanning millions of lines of code, under a roughly $1,000 token budget and one-day completion target. Findings include an estimated 8,500 monthly API calls saved, a leadership-ratified migration plan, and incidental discovery of a production defect. The analysis concludes that governance throughput, not code generation capacity, is the dominant constraint on modernization velocity in enterprise settings, with implications for how organizations deploy agent fleets.

1. Introduction

The dominant narrative surrounding AI-assisted software engineering centers on code generation speed. In large, regulated enterprises, however, writing code was rarely the primary bottleneck. The inner loop - coding, building, and local testing - has accelerated substantially with the introduction of agentic coding tools. The outer loop - assembling appropriate stakeholders, synthesizing institutional knowledge, securing approvals, and documenting decisions - has not kept pace. As framed in the source material, organizations "find ourselves seemingly stuck within the bounds of this outer loop, which is to say everything else that happens other than the coding and the building."

This constraint is pronounced at GEICO, an insurance carrier for which risk management is not a discrete phase but a property embedded at every stage of delivery. The organizational consequence was measurable: engineers retained 25% or less of their bandwidth for major (V1→V2→V3) initiatives after absorbing patches and minor upgrades. Using semantic versioning as a lens for prioritization, GEICO observed that major work was rate-limited not by implementation effort but by the speed at which the right people could be convened and context synthesized.

The central thesis explored here is that the outer loop itself can be accelerated by AI - not by removing human judgment from decisions, but by deploying agent fleets to prepare, structure, and pressure-test the artifacts on which human decisions depend. This analysis reviews the problem context, the war room construct, three domain experiments, the supporting technical architecture (FleetKit), and the operational lessons derived from token spend, model selection, and subagent behavior.

2. Background and Related Work

Three conceptual inputs underpin the approach. First, Anthropic's code modernization plugin supplied a taxonomy distinguishing uplift (incremental extension of existing assets), transform (new interfaces atop existing logic), and reimagine (full rearchitecture). This taxonomy served as a scoping mechanism, calibrating expected effort and governance intensity to the class of change undertaken.

Second, FleetKit - a structured, markdown-based collection of rules, protocols, and instructions optimized for the Claude harness referred to as Fable - provided the orchestration layer purpose-built for architecture transformation work. FleetKit required advanced model capability (cited as Opus 4.8 or above) to handle steering instructions operating one level above conventional system prompting.

Third, the war room metaphor, conventionally associated with reactive incident management, was repurposed for proactive modernization planning, with agents functioning as research and synthesis staff rather than autonomous decision-makers. This composition reflects an underlying premise: "Developing software looks less like playing in a canvas and more like working in a factory," implying a shift toward structured, process-driven agent deployment rather than ad hoc prompting.

3. Core Analysis

3.1 The War Room Construct

The war room combined three elements: a roster of human decision-makers and subject-matter experts, a human "driver" responsible for operating AI tooling, and an AI-native productivity suite encompassing communications channels, an interactive design studio, collaborative tooling, and code repositories. These elements were wired together via workflows, agent skills, Model Context Protocols (MCPs, including enterprise-specific instances), and fleets of agents assigned to research, fact-checking, and synthesis tasks. The explicit goal was accelerating delivery of agent-callable APIs into legacy core claims systems comprising millions of lines of code - work previously constrained by human coordination overhead rather than coding capacity.

3.2 Three Bounded Experiments

The uplift/transform/reimagine taxonomy was applied across three domains, each operating under an approximate $1,000 token budget with a one-day end-to-end target for domain mapping and deployment planning.

In the rentals domain (uplift), an agent extended an existing API and identified an optimization enabling an estimated 8,500 calls per month in savings through more current rental information retrieval. In the damages domain (transform), agents constructed new agent-callable APIs to unlock a larger downstream workflow requiring damages data - illustrating the construction of new interfaces over existing legacy logic. The salvage domain (reimagine) represented the highest complexity case: agents produced a fully vetted migration plan, including API specifications and design mockups, which was subsequently ratified by both product and technical leadership. Notably, the agent fleet also identified a production bug during this process, an incidental but consequential finding given the domain's complexity.

3.3 Technical Architecture: FleetKit and Agent Roles

The architecture separated vendor-provided components (harness, model, FleetKit, code modernization plugin) from GEICO-owned components (knowledge base, tools, methods, and governance judgment). FleetKit operationalized a nine-step process beginning with a "booting" step, proceeding through a dispatch of "leader" agents for research, placement of proposals into a human-in-the-loop approval queue (implemented via GitHub issues), and subsequent dispatch of "builder" agents once approvals were secured. Human approval gates - referred to as "worksheet" milestones and visually marked in orange - decreased in frequency toward process completion, reflecting increasing confidence as artifacts matured.

Two distinct agent types were deployed: cloud-based "code divers" tasked with crunching and graphing legacy domain-specific languages, and local agents operating with laptop and MCP access for enterprise-specific data retrieval. The driver's laptop functioned as the runtime environment for this nine-step process. A task-tracking tool called beads helped agents manage work within their own agent space, while the code modernization plugin performed deterministic analysis verifying that prior steps had produced required artifacts before progression.

3.4 War Room Artifacts and Collaboration Patterns

At the outset of each mission, an agent created or adopted a dedicated communications channel, inviting relevant war room members based on domain relevance. At each milestone, agents produced either auto-saved, fillable HTML worksheets or design documents such as draw.io or Figma diagrams. When agents identified gaps in product or technical knowledge, they proactively mentioned specific individuals from the roster to solicit input. Notably, the quality of these artifacts prompted favorable responses from participating teams - including instances where reviewers remarked "great question" in response to agent-generated worksheets - and some diagrams were retained by teams for continued use beyond the experiment.

4. Technical Insights

Several implementation-level findings merit attention for practitioners considering similar deployments. First, token budget discipline requires active management: agents should not be permitted to configure their own cron jobs, as doing so triggers a new session launch at each interval, rapidly depleting budget. Second, prompt caching yielded efficiency gains exceeding initial expectations, suggesting that repeated context across agent invocations can be exploited more aggressively than commonly assumed. Third, subagent behavior exhibited quirks requiring active monitoring rather than unattended operation, indicating that current agent fleets are not yet suited to fully autonomous long-running operation without human oversight checkpoints.

Fourth, and significant for architecture decisions, benchmarking revealed that Sonnet 5.5 produced legacy-style code as well as or better than Fable in certain tasks, underscoring that model selection - rather than harness sophistication alone - materially affects output quality. This finding complicates a straightforward recommendation to standardize on any single harness-model pairing and instead favors empirical benchmarking per task type.

5. Discussion

These findings suggest that in regulated enterprises, the primary constraint on AI-driven modernization is not code generation capacity but governance throughput - the rate at which proposals can be synthesized, reviewed, and approved by accountable humans. The war room model addresses this by having agents produce decision-ready artifacts (worksheets, diagrams, migration plans) rather than merely code, shifting agent labor toward the preparatory and synthesis work that traditionally consumed human coordination time.

This has implications beyond GEICO's specific context. Organizations with poor or slow decision-making processes - explicitly identified as a target profile - may be well suited to this approach, provided token costs are controlled and the architecture is customized to institutional knowledge systems. A notable gap is the current reliance on flat markdown files for context and search; the source material identifies graph-based context tools and audio transcription of meetings as future directions to reduce context loss and improve retrieval efficiency, suggesting current implementations remain an early iteration rather than a mature pattern.

6. Conclusion

GEICO's war room experiments demonstrate that agent fleets can meaningfully compress the outer loop of software delivery - not by eliminating human decision-making, but by accelerating the preparation of artifacts underlying those decisions. Across uplift, transform, and reimagine experiments, measurable outcomes included quantified API call savings, a leadership-ratified migration plan, and an incidentally discovered production bug, all achieved within approximately $1,000 in token spend per domain and a one-day timeframe. Practitioners should weigh model selection carefully against harness choice, guard against runaway token consumption from agent-configured automation, and consider targeting organizational units with weak existing decision processes as initial deployment candidates for this pattern.

Sources

• One Day, $1,000 in Tokens: An AI War Room for Legacy Code - Ford Prior, GEICO - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub