Agents Made Code Free. Maintaining It Didn't Get Cheaper. with Dmitry Buykin, Maersk

Managing entropy in large AI-agent-written codebases requires a weekly process of small, evidence-based sweeps guided by six questions (time, space, reality,

By Sean Weldon

Agents Made Code Free. Maintaining It Didn't Get Cheaper.

Abstract

This synthesis examines a practitioner account of managing entropy in a large-scale, agent-written codebase spanning 110 countries, where prose instructions constitute roughly three-quarters of the system's behavioral specification. The central thesis is that coding agents reduced the marginal cost of writing software by a larger factor than they reduced the marginal cost of producing disorder, and that a one-time destructive rewrite fails to resolve this because generative agents regrow deleted structure. The proposed alternative is a recurring weekly discipline organized around six concerns - time, space, reality, power, soul, and control - supported by techniques including mutation testing adapted to natural language, a prose linter validated against a code ledger, a blast-radius metric, ratchet metrics, and two-agent verification. The principal finding is that verification effort, not generation effort, becomes the dominant and binding cost of agent-driven development.

1. Introduction

Coding agents have made the marginal cost of producing software approach zero. This synthesis examines the claim, advanced by a practitioner managing a production system across 110 countries, that the marginal cost of producing entropy - the accumulation of unjustified variation, duplicated truth, and unreachable logic - fell even faster. The system under discussion is unusual: approximately 10,000 scenarios of prose instructions make up an estimated three-quarters of its specification, with conventional code comprising the remainder. Two classes of agents interact within it, one authoring instructions and another executing them in production, each task confined by default to a slice - one process in one country.

The practitioner's first response to accumulating mess was a large deletion of agent-generated tests and unused boilerplate, characterized by the speaker as a Thanos-like erasure. This did not hold. Because the agents that produced the original mass continued operating, the deleted structure regrew. This failure establishes the paper's central dilemma: either disorder is allowed to compound unchecked, or periodic destructive consolidation breaks behavior that users depend on.

"Drown in chaos or break what people use. In here, we'll build the third option."

The third option, and the subject of this analysis, is a weekly cadence of small, evidence-bearing sweeps organized around six questions. This paper summarizes the conceptual apparatus underlying that cadence, extracts the technical mechanisms reported to implement it, and situates the result within the broader claim that AI-driven development shifts the binding constraint from authorship to verification.

2. Background and Related Work

The method draws on three established practices, each adapted to a context where the majority of the specification is unstructured prose rather than code. Mutation testing, normally used to assess whether a test suite is adequate by deliberately injecting faults into code and checking that tests fail, is repurposed here for natural-language instructions: a sentence is corrupted and the system is observed for a resulting test failure. Conway's Law, which holds that system structure mirrors organizational structure, is extended to a claim about correction rather than design - the system is reported to mirror whoever most recently corrects it. Decision-table coverage supplies the invariant used to validate refactors: each branch of a decision table must be exercised exactly once after a transformation, with a reported regression involving a silently dropped branch for amounts exceeding a threshold.

A further background claim concerns the limits of agent capability in complex systems:

"Agents can't build a complex system on their own. They fix what is easy to see and break consistency on the way."

This framing establishes why human and programmatic oversight remains structurally necessary even as authorship is automated.

3. Core Analysis

3.1 Time and Space: Establishing What "Right" Means

Because the system is predominantly prose, conventional correctness oracles are unavailable for most of its specification. Code admits tests that specify what must survive a change; prose does not.

"For code, tests can tell you. For prose, only people can."

Mutation testing applied to prose - deliberately breaking a sentence and watching for a failing test - revealed that almost none of the resulting mutations were caught, indicating the prose layer was effectively unverified. The practitioner's response was to make the links between business rules and underlying systems explicit in writing, so that domain experts could check logic that developers alone would otherwise own, following the operating principle "business tells us what, we work out how." Correctness itself is held to be socially rather than mechanically determined: no single person knows the full specification, so a group of experts adjudicates it, with a single canonical copy of each rule maintained to prevent drift.

3.2 Reality: Distinguishing True from True-Looking

A described production bug illustrates the risk of prose-code divergence: an instruction stated a limit of 1,500 while the corresponding code enforced 1,000, a discrepancy of the same order of magnitude that was tolerable in one context and excessive in another, with no error raised. This motivated construction of a prose linter, which checks every changed instruction file against a ledger derived from code, run by the instruction's author at the time of change.

A second concept, the walk - the path an agent takes through a sequence of prose instructions - governs when behavior should be converted from prose to code. When a walk repeats and produces agreement across countries, it is extracted into a reusable workflow tool, and the corresponding prose is deleted. An early implementation of this conversion made outcomes worse, because leftover prose continued to name the steps now encapsulated in the tool, causing the agent to re-execute them. The resulting rule is that a walk is only promoted to code once repetition and cross-country agreement are both observed; otherwise, judgment is left in prose wherever an expert can still read and verify it. This logic has been bidirectional in practice: hundreds of lines of expressions were moved from code into prose because experts could not read them, while numerous regular expressions were moved from prose into code once they proliferated beyond a readable scale.

3.3 Power and Soul: Controlling Context and Judgment

Control over agent behavior is reported to begin with controlling what the model is permitted to see. Allowing a model to select its own context caused it to load excessive information and fabricate a nonexistent pricing check. The corrective architecture places context selection under runtime control rather than model control: for a representative task, the runtime supplies only three relevant facts drawn from the order, referenced through short handles rather than full data, minimizing context size while the runtime independently validates any resulting write. Writes are gated twice - once during development, and again at write-time in production - with five sequential validation checks ("walls") preceding a write and a sixth confirming it landed, backed by a journal guaranteeing exactly-once execution on retry.

A parallel concern addresses human judgment capacity. The practitioner initially reviewed every change personally, which caused the system to absorb his individual habits and biases - an instance of Conway's Law applied to correction rather than design. As change volume exceeded what manual review could sustain, the practitioner reports using his own fatigue and inability to explain a change as a signal of lost control, regardless of dashboard status. The response was to build instruments measuring derivatives of change rather than reading every diff, including a blast-radius metric validated by replaying it against several thousand historical sessions before deployment, where it fired on fewer than 1% of cases.

4. Technical Insights

Several reported techniques generalize beyond this specific system. First, mutation testing can be extended from code to natural-language specifications by deliberately corrupting a statement and confirming that some downstream check fails; the absence of a failure indicates an unverified specification path. Second, a linter can be built for prose by validating changed instruction text against a structured ledger derived from code, catching numeric or logical divergence such as mismatched thresholds. Third, a ratchet metric - a count permitted only to decrease - provides a low-maintenance way to track the retirement of undesired patterns ("bridge line" violations) over time without requiring continuous manual audit. Fourth, verification itself is reported to consume approximately 90% of agent working time, a trade-off suggesting that compute budgets for agentic systems should weight checking at least as heavily as generation. Fifth, a two-agent verification protocol - an independent checker agent working from a checklist without access to the author agent's reasoning - is reported to prevent the author from talking the checker into false agreement, a mitigation against sycophantic self-verification. A limitation implicit in this design is that it still requires a correct checklist; the method does not eliminate the need for human-authored evaluation criteria, only the need for human execution of them at scale.

5. Discussion

The central synthesis is that automating authorship does not reduce total engineering effort; it relocates effort from writing to checking. The reported finding that agents spend close to 90% of their time verifying rather than producing change, and that a backlog audit found only about a quarter of apparently covered items could actually be confirmed closed, supports the claim that verification is now the binding constraint on throughput in agent-assisted systems. This has implications for how organizations allocate engineering capacity: tooling investment aimed solely at generation speed may have diminishing returns relative to investment in oracles - mutation-tested prose, replay-based evidence, independent review agents - that make verification itself faster and more trustworthy.

A further implication concerns organizational theory. The extended Conway's Law claim - that a system mirrors whoever corrects it, not merely whoever designs it - suggests that bias propagation in agent-written systems is a function of review practice, not just original authorship, meaning rotation or diversification of reviewers may be a previously underweighted lever for controlling systemic bias. An open question, not resolved in the source material, is how this method scales when the correcting population is itself distributed across many individuals with no single point of fatigue to serve as an early warning signal, as it did in this single-practitioner account.

6. Conclusion

This analysis has summarized a practitioner method for governing entropy in a hybrid prose-and-code system under heavy agentic authorship, organized around weekly, evidence-based sweeps rather than either unchecked accumulation or periodic destructive rewrites. Its principal contributions are the adaptation of mutation testing to natural-language specifications, a prose linter grounded in a code ledger, a validated blast-radius metric, ratchet metrics for tracking long-term cleanup, and a two-agent, checklist-based verification protocol. The practical takeaway for teams operating agent-written codebases is that verification infrastructure - not generation tooling - should be treated as the primary engineering investment, and that any oversight process, human or automated, is itself a vector through which systemic bias and behavior are transmitted into the system it governs.


Sources


About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub

Sean Weldon builds apps and custom software at 22Atlas.

Start your build

More about Sean