Your Spec Has Bugs. Catch Them Before Your Agent Codes. with Rajath Mahendrappa Kotyal, AWS

Before handing specifications to a coding agent, teams should formally verify them using automated reasoning (LLM refinement + SMT solver checking + sampled

By Sean Weldon

Your Spec Has Bugs: Formally Verifying Specifications Before Coding Agents Act On Them

Abstract

Natural-language specifications handed to coding agents frequently contain defects invisible to human readers but materially damaging to generated code. This synthesis examines a pipeline - presented as the Keto Specification Analyzer by an AWS automated reasoning group - that verifies specifications before a coding agent ever sees them. The approach layers three mechanisms: LLM-driven refinement of specification wording, formalization of English rules into SMT (Satisfiability Modulo Theories) formulas checked by a satisfiability solver, and repeated sampling of translations clustered by logical equivalence to surface divergent interpretations. Cited research reports a 20-40% accuracy drop under unclear prompts, that 60-90% of superficially valid generated code is nonetheless wrong, and 2x more regressions when such code ships. The architecture's governing principle is that the human user, not the model or the solver, remains the sole authority on specification correctness.

1. Introduction

Agentic software development has shifted the point of failure upstream. Where traditional engineering subjects design documents to human review before implementation, prompt- and agent-driven workflows commonly pass prose specifications directly to a code-generating model with no intervening check. Defects originating in the specification are then silently inherited by the codebase, often without any visible symptom until regression or production failure.

Four categories of specification defect are relevant to this discussion. Ambiguity and wrong level of detail occur within a single sentence - a rule admits more than one reading, or is stated at a granularity the implementer cannot act on. Conflicts and gaps occur across sentences - a conflict exists when one rule contradicts another, and a gap exists when no rule covers a reachable state.

The illustrative case is an e-commerce order system. Rule 2 states: "when an order is submitted and inventory is unavailable, place the order as a back order." Rule 3, elsewhere in the same document, states: "no order shall be placed as a back order." Each sentence is independently well-formed and plausible. Read together, as an LLM necessarily reads them, they are contradictory - and no amount of fluency in the generating model resolves an inconsistency present in its input.

The central thesis is that specifications should be formally verified prior to code generation, using a composed process of LLM refinement, solver-based formal checking, and sampled-translation clustering, escalating to a human whenever automated methods cannot adjudicate. This analysis covers the motivating evidence, the three-step verification mechanism, the sampling-based ambiguity detector, the human-in-the-loop resolution model, and the stated limitations of the approach.

2. Background and Related Work

The motivating case is quantitative. Public research cited in support of the approach reports a 20-40% drop in accuracy for LLM output generated from unclear prompts. More consequentially, 60-90% of code that appears valid is still wrong once written - meaning surface plausibility is a poor proxy for correctness. When such code is deployed, a 2x increase in regressions has been observed relative to code built from unambiguous specifications. Together these figures suggest that reviewing generated code after the fact is an inefficient control point: defect rates are high, defects are camouflaged by superficial validity, and remediation cost is incurred downstream rather than at the point of origin.

The technical foundation draws from automated reasoning rather than machine learning per se. SMT extends Boolean satisfiability (SAT) with decision procedures over background theories, allowing logical statements expressed in domains like arithmetic or set membership to be checked for satisfiability. SMT-LIB is the standard input format for such solvers. Applying this machinery to natural-language specifications requires an intermediate translation step - from English rule to formal formula - which is the role the LLM plays in this pipeline, with the solver providing the actual correctness guarantee rather than the model's own judgment.

3. Core Analysis

3.1 The Three-Step Refine-Formalize-Verify Process

The pipeline begins with the least invasive intervention: the LLM is asked to refine and clean up the specification's wording on its own. If this resolves the issue, no further action is needed. If refinement fails, the pipeline escalates to formalization - English-language rules are translated into SMT-LIB formulas and passed to a SAT/satisfiability solver. The solver returns a Boolean result: valid, meaning no contradiction is detected and no action is needed, or invalid, meaning an ambiguity or conflict exists, at which point the offending sentence is flagged and returned to the LLM to recheck and reiterate. The back-order example above is a direct instance of this: the natural-language rule is translated into a mathematical formula, and the solver can determine that rules 2 and 3 cannot both hold simultaneously.

This design reflects an important asymmetry: refinement is cheap and attempted first, while formal checking is reserved for cases where wording cleanup is insufficient to resolve the underlying logical conflict. The solver's Boolean output also establishes a hard stopping criterion - rather than relying on the model's confidence or stylistic polish, verification proceeds only once a formula is proven valid.

3.2 Deeper Ambiguity Detection via Sampled Translation Clustering

Formalization alone can fail if the LLM's translation into SMT-LIB is itself an artifact of how the model interpreted an ambiguous rule, rather than a faithful representation of the specification. To separate model-inherent variability from genuine specification ambiguity, the same rule is translated 7-10 times and the resulting formulas are clustered by logical equivalence. Distinct clusters correspond to distinct sets of assumptions the model made when interpreting the same English text.

The clustering outcome then drives one of three actions: accept an interpretation (if clustering converges on a dominant, consistent reading), ask the user to clarify (if multiple clusters are roughly comparable in weight), or abstain from implementing the rule entirely (if no stable interpretation emerges). This mechanism distinguishes the pipeline from a simple "ask the LLM to be more careful" approach - it treats interpretive divergence as measurable evidence rather than an unobservable property of the model's internal state.

3.3 Human-in-the-Loop Resolution and the Oracle Problem

A central design commitment is that the system never allows the LLM to silently resolve genuine ambiguity by guessing. The illustrative case is the instruction "remove the record," which is compatible with at least two materially different implementations: deleting the record outright, or retaining it while hiding it from users. Rather than let the model choose, the system surfaces both concrete options to the user for explicit selection.

This reflects the stated position that "there is no guarantee from an LLM regarding correctness - only the user can provide that guarantee." The justification given is that correctness depends on implicit domain knowledge the model does not possess and cannot infer purely from the text of the specification. Consequently, the user - not the LLM, not the solver, not the tool as a whole - is treated as the sole oracle for specification correctness. Conflicts (rule 2 vs. rule 3) and gaps (e.g., no rule addressing an "idle order" state) are both routed through this same human-adjudication channel once automated methods have identified them.

4. Technical Insights

Several implementation-relevant findings emerge from this architecture. First, formal verification is applied as a pre-filter, not a replacement for human review - the solver's role is to flag inconsistency, not to produce a final interpretation. Second, the translation-sampling step (7-10 samples per rule) functions as a diagnostic for whether an ambiguity is intrinsic to the specification text or an artifact of a single stochastic model run; this is a meaningfully different signal than single-shot translation confidence. Third, logical-equivalence clustering, rather than textual similarity clustering, is necessary because superficially different SMT-LIB formulas can be semantically identical, and superficially similar ones can diverge.

A notable trade-off is cost: translating each rule multiple times and running solver checks on each is more expensive than a single LLM pass, and the approach explicitly accepts this cost in exchange for earlier defect detection, consistent with the premise that spec-stage defects are cheaper to fix than code-stage defects. A further limitation, acknowledged directly, is that evidence for applying formalization techniques within agentic systems generally is still thin - this is presented as an emerging area rather than settled practice.

5. Discussion

The approach situates specification verification as a discrete engineering phase analogous to design review, rather than treating prompt quality as something to be improved reactively after code generation fails. This has implications for how coding-agent workflows are structured: it suggests inserting a formal gate between specification authoring and code generation, with abstention as an acceptable and even desirable outcome when the specification itself is underdetermined.

A notable gap is the absence of broader evidence on how such formalization techniques generalize across agentic systems beyond the presented case studies - the field is described as still emerging, with limited external validation. This suggests the architecture should currently be understood as a promising mechanism demonstrated on narrow examples rather than a validated general solution.

The approach also connects to a broader industry pattern of inserting verification layers between natural-language input and generated artifacts, echoing concerns in the same vein as type checking or static analysis, but applied one layer earlier - at the specification itself rather than at the generated code.

6. Conclusion

This analysis has examined a three-stage pipeline - refinement, SMT-based formalization, and sampled-translation clustering - for detecting ambiguity, conflict, and gaps in specifications before they reach a coding agent, alongside a design principle that reserves final correctness judgments exclusively for the human user. The practical takeaway is straightforward: specifications should be checked with the same rigor historically reserved for design documents, using a workflow where the LLM translates, the solver verifies, and the human decides whenever automated methods cannot. Where no clear answer exists, the system should employ LLM-as-judge heuristics or abstain outright rather than silently guess. Given the thin evidentiary base for agentic formalization generally, teams adopting this pattern should treat it as an early-stage technique warranting careful validation within their own domains.


Sources


About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub

Sean Weldon builds apps and custom software at 22Atlas.

Start your build

More about Sean