'Orchestras, Not Factories: How the Fastest Builders Work - Charlie Holtz, Conductor'

The fastest builders in AI engineering follow a set of principles - staying near the frontier without over-optimizing, maintaining human review zones, centrali...

By Sean Weldon

Orchestras, Not Factories: How the Fastest Builders Work

Abstract

This synthesis examines operating principles derived from practitioner experience building Conductor, a desktop application for orchestrating multiple coding agents. The central thesis holds that velocity in AI engineering depends less on tooling sophistication than on disciplined decisions about where effort is invested: remaining near the model frontier without over-engineering personal workflows, demarcating codebase regions requiring human review, centralizing organizational context into a queryable substrate, granting agents persistent execution environments, and treating human-agent collaboration as orchestration rather than industrial automation. Evidence is drawn from architectural decisions including a Postgres-backed internal agent ingesting Slack, Discord, and meeting data, and a migration from local git worktrees to cloud sandboxes. Practical implications concern context engineering, quality governance, and interface design for multi-agent teams, suggesting that competitive advantage accrues to teams that correctly identify which investments compound versus which will be subsumed by default agent harnesses.

1. Introduction

The rapid release cadence of frontier coding agents has produced a methodological gap: practitioners possess increasingly capable tools but lack settled conventions for deploying them at team scale. AI engineering, understood as the practice of building software with and around autonomous coding agents, has generated a proliferation of bespoke workflows whose marginal value is often unexamined by their adopters.

Two terms require early definition. Slop denotes low-quality, unreviewed model-generated output that accumulates in a codebase and progressively degrades its coherence, as evidenced by Conductor's own experience of multiple rewrites. Alpha, borrowed from quantitative finance, denotes proprietary information about a specific user base or codebase that general-purpose models do not possess, and which therefore justifies targeted workflow investment.

This paper synthesizes a principle set articulated under the mnemonic STICFO: Stay near the frontier, don't beaT the market, create slop-free zones, Feed the beast, free-range agents, and Orchestras, not factories. Each addresses a distinct failure mode: informational lag, workflow over-optimization, quality degradation, context starvation, execution constraint, and dehumanizing interface design, respectively. The central question guiding this analysis is: under conditions of rapid model improvement, which engineering investments compound over time, and which are rendered obsolete by the next default release? Section 2 establishes background; Section 3 analyzes the principles thematically; Section 4 extracts implementation guidance; Section 5 discusses broader implications.

2. Background and Related Work

The principles emerged from a specific development history. Conductor consolidates management of multiple coding agents - including Claude Code and Codex - within a single interface, replacing the common pattern of numerous parallel terminal windows. Its origin is instructive: heavy use of Claude Code within a prior product, Chorus, led to the practice of cloning a repository five times to enable concurrent agent work, and subsequently to the discovery of git worktrees as a cleaner primitive. The product thus arose directly from frontier usage rather than from conventional market analysis, an observation that anchors the first principle.

Two conceptual borrowings structure the analysis. The first is the efficient market hypothesis, applied analogically to workflow optimization: if a technique improves outcomes for all practitioners uniformly, it will eventually be absorbed into the default harness shipped by major laboratories such as Anthropic or OpenAI, rendering private investment in it unrecoverable. The second is a critique of the software factory metaphor, which is explicitly rejected and compared to the earlier and similarly unsuccessful feature factory terminology of the preceding decade.

3. Core Analysis

3.1 Frontier Proximity and the Limits of Optimization

The first two STICFO principles jointly govern where effort should be directed. Staying near the frontier - trialing tools such as Claude Code or Slash Go on release day - is presented not as a productivity ritual but as a generative practice: Conductor itself emerged from this behavior rather than from deliberate planning. Practitioners who rely on social-graph trickle-down are described as perpetually three to six months behind, given the pace of change.

This principle is counterbalanced by "don't beat the market," which cautions against midwit meming - excessive optimization of workflow at the expense of substantive output, captured in the observation that one should not be "the person who has an amazing Emacs setup but doesn't actually get stuff done." The operative heuristic is to ask why a given workflow (e.g., Ralph loops) is not already the default; if it would benefit all users equally, it is likely to be absorbed into lab-provided harnesses, making private investment wasteful. Investment is justified only where genuine alpha exists, illustrated by Conductor's targeted work on React query optimization for chat rendering performance - a problem specific to its architecture that general models cannot solve from training data alone.

3.2 Governance: Slop-Free Zones and Centralized Context

Where the first pair of principles concerns effort allocation, the second pair concerns quality and information governance. Slop-free zones are codebase areas designated for mandatory human review, a practice adopted only after Conductor had to rewrite its application multiple times due to unchecked AI-generated degradation. Concrete implementations include mandatory human review of migration files in CI and treating Slack messages as inherently human-authored and slop-free. Considerable effort is additionally invested in documentation artifacts - CLAUDE.md files and skills files - framed through the analogy of information "whispered into a new intern's ear every day," warranting the same care as onboarding a human employee.

Complementing this is "feed the beast," operationalized through the Conductor Internal Agent (CIA), a centralized Postgres database ingesting Slack messages, Discord bug reports, and meeting recordings. The stated mechanism is deliberately simple: "put everything in a database and then give your agent a SQL tool and let it handle the rest." This reflects a broader architectural preference for centralizing organizational context rather than relying on agents to reconstruct institutional knowledge from fragmented sources.

3.3 Execution Freedom and Collaborative Interfaces

The "free-range agents" principle addresses execution constraints. As models improve and can sustain longer-running tasks, confining agent execution to a single laptop session becomes a bottleneck; agents require sandboxed space to explore and persist independent of whether a local machine remains open. The forthcoming Conductor version reflects this by migrating task execution from local git worktrees to cloud sandboxes, which additionally enable real-time collaborative viewing of teammates' (and agents') active workspaces. A demonstrated extension - a Telegram bot ("Lord Crandon") with API access to spawn new workspaces autonomously - illustrates agents initiating their own execution contexts rather than operating solely within human-defined boundaries.

4. Technical Insights

Several implementation-level findings merit emphasis for practitioners designing similar systems:

A trade-off is evident in the slop-free zone strategy: broad application would reintroduce the bottlenecks AI tooling is meant to relieve, while insufficient application permits degradation, as Conductor's rewrite history demonstrates. The boundary-setting decision is therefore treated as a continual judgment call rather than a fixed policy.

5. Discussion

These principles collectively suggest that competitive differentiation in AI-assisted engineering is shifting away from raw tool access - which is rapidly commoditized - toward judgment about resource allocation and information architecture. The "don't beat the market" principle, in particular, implies that much workflow engineering currently practiced across the industry may have a short shelf life, as labs progressively internalize effective patterns into default harnesses.

The rejection of the "software factory" metaphor, paralleled with the prior decade's "feature factory" terminology, signals a broader concern about interface design as agent capability scales. The orchestras, not factories framing positions the human role as one of synthesis and direction across intermingled teams of humans and agents, rather than supervision of an automated pipeline. This raises an open question not fully resolved in the source material: as agents gain the ability to spawn their own workspaces and operate with greater autonomy, the mechanisms by which human "conductors" maintain situational awareness across an expanding set of concurrent agent activities remain underspecified.

6. Conclusion

This analysis has synthesized six interrelated principles - frontier proximity, selective optimization, quality governance, context centralization, execution freedom, and collaborative framing - into a coherent account of how fast-moving AI engineering teams allocate effort. The practical takeaway is that sustainable velocity depends on distinguishing investments with durable alpha from those destined for commoditization, while simultaneously building structural safeguards against quality degradation and architecting for agent persistence and collaboration. Future work might examine how these principles generalize beyond coding-agent contexts and how human oversight mechanisms should evolve as agent autonomy increases.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub