AI Writes Code Faster Than Uber Can Merge It with Preetam Dwivedi & Manjari Akella, Uber

Uber scaled its Summit Q merge queue and Stowpipe post-land validation system to handle the surge in AI-generated code changes, trading some premerge safety

By Sean Weldon

AI Writes Code Faster Than Uber Can Merge It: Scaling Merge Infrastructure for AI-Generated Code

Abstract

Large monorepos depend on merge queues to guarantee that every commit landing on trunk is tested against the exact state it will land into. This synthesis examines how a surge in AI-generated code changes forced Uber to adapt Summit Q, its merge/submit queue, and Stowpipe, its post-land validation system, trading premerge safety for throughput through admission gating, deferrals, and conflict relaxation. Drawing on reported production metrics, the analysis shows request volume rising 44% in two months, P95 land time degrading from 30 minutes to 3-4 hours, and successive interventions restoring throughput to roughly 300 lands per hour - a 233% increase over baseline - while Stowpipe absorbed the resulting risk of trunk breakage through exhaustive post-land validation. The findings suggest that at sufficiently high submission rates, hybrid validation strategies outperform strictly green-trunk policies, with conflict-graph relaxation identified as the dominant throughput lever.

1. Introduction

A merge queue serializes the path from an approved pull request (PR) to trunk, ensuring each candidate is built and tested against the precise trunk state it will occupy. The canonical failure case motivates this architecture: if PRs A and B are independently validated against a green trunk and then merged sequentially, B is never tested in the presence of A's changes. Semantic conflicts invisible to textual merge checks can therefore reach trunk undetected.

This problem compounds with scale. Uber operates six-language monorepos serving more than 5,000 developers, absorbing approximately 36,000 commits and 27,000 PRs per week. At that throughput, trunk breakage is not an isolated event but a continuous organizational cost, since every broken build blocks downstream developers until it is resolved.

The central question this analysis addresses is what happens to merge-queue architecture when submission volume rises sharply enough to exceed premerge validation capacity - a condition precipitated by the growth of AI-generated code changes. The source material frames this as "Uber's journey from going to a fully green trunk to somewhat broken trunk," a deliberate and measured departure from exhaustive premerge guarantees. The analysis proceeds from the underlying mechanisms of Summit Q (Section 3.1), through the specific load shift caused by AI-generated changes and the interventions deployed in response (Sections 3.2-3.3), to Stowpipe's compensating role (Section 3.4) and broader implications (Sections 4-5).

2. Background and Related Work

Two concepts underpin the system design. Speculation allows a queue to validate multiple hypothetical trunk states in parallel - A alone, B alone, A+B together - landing whichever combination is both validated and consistent with queue order. Target analysis uses the build graph to identify which targets or services a change touches; changes with disjoint target sets cannot interact semantically and can therefore be validated concurrently without weakening correctness guarantees.

Summit Q's design objective is stated directly in the source material: "We want to land the change that is going to keep the trunk green as a whole." This objective holds under normal load but becomes harder to satisfy as submission volume grows, since speculative build combinations scale combinatorially with queue depth. Both Summit Q and Stowpipe are released as open source, situating this work within the broader ecosystem of publicly documented merge-queue implementations rather than purely internal tooling.

3. Core Analysis

3.1 Baseline Mechanisms and the AI-Driven Load Shift

Under normal conditions, Summit Q combines target analysis, probabilistic speculation (using historical build data and change characteristics to compute a success score prioritizing the most likely-needed builds), and bypassing of large changes (allowing smaller, faster-building changes to skip queue position ahead of slower ones) to maximize throughput without compromising trunk integrity.

This equilibrium broke down as AI-generated changes entered the pipeline. Request volume rose from approximately 90 requests per hour in January 2026 to 130 requests per hour by March 2026, a 44% increase in two months attributed primarily to AI-generated changes. The consequence was not trunk breakage but latency: P95 land time degraded from 30 minutes to 3-4 hours. The queue preserved correctness by absorbing the excess load as wait time, at the cost of developers waiting hours to land otherwise-ready changes.

3.2 Admission Gating and Deferrals

The first intervention, admission gating, classifies incoming requests as "small" or "expensive" based on target analysis, admitting small changes during peak hours while rejecting expensive ones and forcing their authors to retry later. This increased throughput over 20%, raising land rate from 130 to approximately 160 requests per hour during business hours. The source material is explicit about the resulting cost: "Developers become the schedulers... you will be particularly targeted in this because the work that you do naturally falls into this expensive change bucket." The scheduling burden shifted from the system to the authors of broad-spanning changes.

Deferrals mitigated this cost by introducing a secondary persisted queue that automatically replays rejected expensive changes back into the main queue once capacity allows, eliminating the need for manual retries. Deferrals did not reduce wait time itself; they removed the operational burden of re-submission.

3.3 Conflict Relaxation

The largest single intervention was conflict relaxation, which breaks overlapping-target "strict connected component" chains that otherwise force serialization. The mechanism scores overlap strength using Jaccard scoring over overlapping targets and newly introduced dependency edges, producing a continuous 0-1 conflict strength value, and drops the weakest edges until a configured parallelism target (for example, 50%) is reached. Edge-dropping decisions are computed once, at a request's arrival time (local assignment), rather than recomputed globally across the queue - a design choice that preserves existing speculative builds and avoids instability from retroactive reordering.

Conflict relaxation raised throughput from approximately 160 to 250 lines per hour, an increase exceeding 50%, the largest single gain among the three interventions. This came at an explicit and acknowledged cost: untested change combinations could now land together, meaning "main can now be broken." This marks the structural transition described in the source material, from a fully green trunk to a probabilistically green one, and creates the new obligation that Stowpipe was built to satisfy.

3.4 Stowpipe: Post-Land Validation

Stowpipe validates commits after they land on main by maintaining a verified main pointer distinct from main's actual head. It tests ranges of newly landed commits for buildability and only advances verified main if validation passes; otherwise verified main remains at the last known-good commit. On detecting a broken trunk, Stowpipe pages on-call engineers, who prepare a fix or revert and land it through Summit Q, after which Stowpipe re-validates until verified main can advance again.

This architecture substitutes exhaustive premerge testing with exhaustive post-land testing for the subset of changes exempted by conflict relaxation, converting a correctness guarantee from a hard precondition into a detect-and-recover loop.

4. Technical Insights

The cumulative effect of these interventions is substantial: peak throughput reached approximately 300 lands per hour while maintaining a 30-minute P95 land time, a 233% increase over the January baseline of 90 lands per hour. Several implementation considerations follow from the mechanisms described:

5. Discussion

These findings indicate that exhaustive premerge validation is not a fixed requirement of merge-queue design but a tunable parameter that can be relaxed under load, provided a compensating detection-and-recovery mechanism exists downstream. Uber's explicit framing - "We chose a hybrid approach to validating the changes... validate whatever matters before and then validating everything at the end to make sure it is still good" - generalizes beyond this specific system: organizations facing superlinear growth in change volume, whether from headcount or AI-generated contributions, may need to accept probabilistic correctness guarantees rather than scaling premerge infrastructure indefinitely.

A notable gap is the absence of reported recovery-time metrics for Stowpipe itself; throughput and land-time figures are well documented, but the latency and frequency of trunk-breakage incidents under conflict relaxation are not quantified in the source material. This matters because the entire hybrid approach's viability depends on recovery being fast and reliable, not merely on-call-driven. Planned future work - batching related changes, moving from repo-wide to per-service green/broken signals, and automating fix/revert recovery - directly targets this gap, suggesting the current system is understood internally as a transitional state rather than a steady-state design.

More broadly, this case illustrates a pattern likely to recur across organizations adopting AI-assisted code generation: infrastructure built for human-paced submission rates encounters capacity limits well before code quality becomes the binding constraint, and the response is architectural (queue and validation redesign) rather than purely a matter of restricting AI tool usage.

6. Conclusion

This analysis demonstrates that Uber's response to AI-driven growth in code submission volume was not to throttle generation but to re-architect its merge pipeline around a hybrid validation strategy: premerge validation where overlap risk is low, admission control and deferral where capacity is the binding constraint, and exhaustive post-land validation via Stowpipe where safety is deliberately relaxed. The reported outcome - a 233% increase in land rate while holding P95 land time at 30 minutes - indicates this trade-off was effective at the measured scale.

The practical takeaway for engineering organizations anticipating similar growth is that target-analysis-based gating and conflict-graph relaxation, paired with a verified-main-style post-land validation loop, offer a reusable template for scaling merge infrastructure beyond what exhaustive premerge testing alone can support. Both Summit Q and Stowpipe are open source, making direct adoption or adaptation of this template a concrete next step for teams facing comparable submission-rate pressure.


Sources


About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub

Sean Weldon builds apps and custom software at 22Atlas.

Start your build

More about Sean