Why Humans Can't Keep Up With AI-Written Code with Tariq Shaukat, Sonar

As AI agents make code generation abundant and cheap, confidence in code quality becomes the scarce resource; software factories must be re-engineered with i

I'll write the complete research paper synthesis now, targeting ~1300 words per the stricter cap.

Abstract

As AI agents make code generation abundant and inexpensive, confidence in the correctness, security, and maintainability of that code becomes the binding constraint on software delivery. This synthesis examines why human-in-the-loop review, the dominant quality mechanism in traditional software factories, fails to scale under agentic throughput, and proposes a multiplicative Code Confidence Formula - intent, guide rails, verification, and evidence - as a structural remedy. Drawing on the Toyota Production System (TPS) as a manufacturing precedent, the analysis documents five empirically grounded limitations of current agentic workflows, including cognitive surrender, large-codebase degradation, and compounding error over long task horizons. Reported deployment results show a 36% reduction in token cost and a 92% reduction in commit-stage issues when the formula's components are engineered into the agentic loop. The practical implication is that confidence must be built into software factories by design, not inspected in after the fact.

1. Introduction

Writing source code was, for decades, the primary bottleneck in software delivery. That constraint has reversed. Published research cited in this analysis shows that introducing asynchronous agents into the development lifecycle increases lines of code written by 2,450% and commit volume by roughly 240%, while releases increase by only 30%. Code has become abundant; the capacity to trust it has not kept pace.

This asymmetry motivates the central question addressed here: if generation is cheap and plentiful, what actually limits throughput of shippable software? The thesis is that confidence - a justified, evidence-backed belief that code is correct, secure, and maintainable - is now the scarce resource, and that it must be engineered into software factories rather than produced through human review at the margins.

Three terms are used precisely throughout. A software factory is the end-to-end system converting intent into deployed software, including tooling, review, and controls. Agentic software development refers to workflows in which autonomous agents plan and execute multi-step engineering tasks with limited human supervision. Guide rails, as distinguished from guard rails, are mechanisms that actively steer agent behavior during execution rather than merely preventing catastrophic failure after the fact.

Code is abundant, confidence is scarce, and the goal is how do we deliver confidence.

The analysis proceeds as follows: Section 2 situates the argument in the manufacturing lineage of software practice. Section 3 analyzes five limitations of agentic development and introduces the Code Confidence Formula. Section 4 presents technical findings and deployment evidence. Sections 5 and 6 discuss implications and conclude.

2. Background and Related Work

The intellectual precedent for this argument is the automotive quality transition of the 1980s. Vehicles of that era were frequently acceptable as individual units but poorly integrated as assemblies - panels and components did not fit together with precision. The often-cited Lexus demonstration, in which a ball bearing rolled uninterrupted across body seams, served as evidence that manufacturing precision was achievable rather than merely theoretical.

Toyota's structural contribution was not better inspection but a redesigned factory: a shift from building for volume and correcting defects downstream, to designing for quality from the start. Relevant TPS principles include prioritizing quality over volume, fixing the system rather than its symptoms, designing for buildability, resolving problems at the point of occurrence, and empowering any line worker to halt production upon detecting a defect.

Much of contemporary software practice - Agile, Kanban, continuous integration, and SRE error budgets - descends from this TPS lineage. These practices assume a human-paced rate of production and human-mediated review as the primary quality gate. The argument developed in this synthesis is that agentic throughput invalidates that assumption, requiring a comparable re-engineering of the software factory itself.

3. Core Analysis

3.1 Five Limitations of Agentic Software Development

The first limitation is that human-in-the-loop review does not scale: the 2,450% increase in code volume and 240% increase in commits against only a 30% increase in releases demonstrates that human reviewers become the rate-limiting stage once agents generate at scale.

The second is cognitive surrender. Humans agree with AI-generated output 93% of the time when the AI is correct, but also agree with it four out of five times when the AI is wrong - indicating that reviewers' judgment is anchored to the agent's output rather than independently formed.

The third is degradation on large, complex codebases. Code translation accuracy measured 91% at 10,000 lines of code but collapsed substantially by 50,000 lines. Token usage variance across tasks on large codebases was measured at 30x (not 30%), attributable to agents expending tokens navigating unfamiliar codebase structure rather than solving the assigned task.

The fourth is error compounding over long-running tasks. Research from Meter indicates that Claude models can complete tasks equivalent to roughly 16 human-hours of work at 50% accuracy, but the accuracy curve flattens sharply as the 80% threshold is approached. This compounding effect is severe at the step level: at 95% per-step accuracy, overall task accuracy falls to 30-36% by the twentieth sequential step, illustrating how small per-step error rates accumulate catastrophically across long agentic chains.

The fifth is the unreliability of LLMs as reward-seeking judges. PayPal AI Research Lab found that 7-17% of verdicts changed across identical replications of the same evaluation question. A related Meta study found that merely asking an LLM judge "are you sure?" changed its verdict 7-34% of the time depending on the model. Because near-zero time-to-exploit now characterizes many security vulnerabilities, unreliable automated judgment materially limits an organization's response capability.

3.2 The Code Confidence Formula

The proposed remedy is a Code Confidence Formula: confidence = intent × guide rails × verification × evidence. The multiplicative structure is deliberate - if any single component is zero, overall confidence is zero, regardless of strength elsewhere.

Intent requires specifying, prior to agent instruction, precisely what is to be built. Guide rails - distinguished explicitly from guard rails, which merely prevent departure from acceptable bounds - actively nudge agents toward correct behavior throughout execution via small, continuous corrections. Verification must be independent of the generating agent, thorough, and must fix identified issues rather than defer them downstream. Evidence follows a "trust but verify" standard: proof that is consistent, auditable, and repeatable, sufficient to satisfy external compliance frameworks such as IT general controls or MISRA.

Trust, but verify. Trust, but understand that it was done with proof that is consistent, audible, and repeatable.

4. Technical Insights

Several implementation-relevant findings follow from the formula's deployment. First, nudging agents within the agentic loop using combined intent-and-guide-rail mechanisms (implemented in the Sonar Vortex product) produced a 36% reduction in token cost and a 92% reduction in issues reaching the commit stage - evidence that correcting agent trajectory during execution is more efficient than post-hoc remediation.

Second, the recommended verification architecture combines deterministic methods (CPU-based, rule-driven static analysis) with agentic mixture methods (GPU-based, LLM-driven semantic evaluation), described as a "gold standard" precisely because it balances precision against cost. Deterministic methods alone miss semantic and contextual defects; LLM-only verification inherits the reliability problems documented in Section 3.1 (variable judge verdicts, inconsistency under repeated questioning). The hybrid approach is a trade-off mitigation rather than a solution to either method's individual limitation.

Third, survey evidence from more than 1,000 SonarQube customers indicates that organizations applying this verification approach are 81% more likely to report a very positive impact on code security, 34% more likely on time to market, and 76% more likely on reduced outage frequency - suggesting the formula's components are not purely defensive but also contribute to delivery speed.

A limitation worth noting: these percentages derive from vendor-sponsored survey and deployment data (Sonar's own product and customer base), which constrains independent replicability relative to the peer-reviewed findings cited for agent limitations (Meter, PayPal AI Research Lab, Meta).

5. Discussion

The findings suggest an industry-wide transition is underway but incomplete. Survey data indicates only 5% of developers currently operate agentic software factories, while 34% expect to within a year (39% in financial services specifically) - implying that the infrastructure described here is nascent rather than standard practice. This gap between adoption intent and current capability is consistent with the thesis that confidence-engineering infrastructure, not agent capability itself, is the binding constraint.

A second implication concerns security. Because agents capable of building software are equally capable of being used to attack it, and because exploit timelines are now near-zero, the governance question extends beyond code quality to insider and systemic risk. This concern was echoed publicly by Satya Nadella, who called for engineering approaches to model containment grounded in deterministic system design, human controls, and reliable operating procedures - a framing consistent with the guide-rails concept presented here.

An open question not resolved by the source material is how the Code Confidence Formula's components should be weighted or prioritized when resources are constrained, given their multiplicative rather than additive relationship. Future work could usefully quantify the marginal return of investment in each of the four components independently.

6. Conclusion

This synthesis argues that agentic coding has inverted the software production bottleneck from generation to verification and trust. The TPS analogy clarifies the nature of the needed response: not faster human review, but factories redesigned for quality from the outset. The Code Confidence Formula - intent, guide rails, verification, and evidence - offers a structured, multiplicative framework for that redesign, with reported deployment results (36% token cost reduction, 92% issue reduction) suggesting meaningful returns.

The practical takeaway for engineering organizations is that agent-era software quality cannot be achieved by adding more human reviewers to an accelerated pipeline; it requires re-engineering the pipeline itself around continuous, independent, auditable verification mechanisms operating at agent speed.

Sources

• Why Humans Can't Keep Up With AI-Written Code - Tariq Shaukat, Sonar - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub