Stop Chatting With Spreadsheets with Francesco Galletta & Felippe Felisola Caso, Monzo

The speed of producing AI outputs is no longer the bottleneck in financial analysis; trust is. Building a trustworthy agentic 'harness' for high-stakes finan

Stop Chatting With Spreadsheets: A Trust Framework for Agentic Financial Analysis

Abstract

This synthesis examines a design framework for agentic harnesses - the engineered scaffolding surrounding a language-model agent - applied to high-stakes financial analysis. The central claim, advanced by practitioners from Monzo and Nubank, is that output generation speed has ceased to be the limiting factor in agentic finance work; the binding constraint is whether human decision-makers can justifiably trust agent-produced conclusions. Five design principles are articulated: shared domain, consistency, evidence, accountability, and verification. Each principle maps to concrete architectural mechanisms, including plan-graph representations with provenance annotation, deterministic versioned calculation code invoked via tool calling, branching sub-conversations, and an adversarial challenge agent operating in a feedback loop. Implications include guidance on when a bespoke harness is warranted over general-purpose coding agents, and how to frame adoption around measurable business outcomes rather than abstract trust in "the AI."

1. Introduction

Contemporary LLM agents can produce financial analyses in minutes that previously required hours of manual spreadsheet manipulation. The thesis advanced here is that this capability improvement has relocated, rather than removed, the bottleneck: as one presenter framed it, practitioners are "past the point of having to spend hours grinding through your keyword to get something that you can trust." The residual difficulty concerns how results connect to, and are accepted by, the humans accountable for acting on them.

Finance is a demanding proving ground for this claim. Errors carry direct monetary consequence, outputs are frequently escalated to senior leadership, and comparability across time periods is itself a correctness requirement, not a convenience. Trust, consequently, cannot be assumed as an emergent property of model capability; it must be engineered.

A terminological distinction anchors the analysis. The harness denotes the deterministic software environment, user interface, state representation, and control flow within which an agent operates - distinct from the agent itself. Familiar components such as unit testing, planning modes, evaluation suites, trace-based observability, and loop or graph orchestration are treated as necessary but insufficient. The presenters, drawing on practice rather than claiming a definitive methodology, target the underlying design principles that determine whether these components compose into something a user can trust, as opposed to hoping a capable agent "does the right thing." Sections 2 and 3 establish context and analyze the five principles; Section 4 extracts implementation-level findings; Sections 5 and 6 discuss scope conditions and takeaways.

2. Background and Related Work

The framework sits adjacent to, but distinct from, spec-driven development as practiced with general-purpose coding assistants. Both treat a written specification as a durable artifact, yet a financial harness must additionally encode epistemic status - whether a quantity is established fact, model inference, or unresolved question - which a conventional spec does not capture.

Two prior patterns are referenced critically. The first is LLM-as-judge, in which a second model statically scores an output; the limitation identified is architectural, since a static verdict provides no path to modify the artifact under review. The second is the pre-automode, line-by-line approval model familiar from early coding agents, which secured oversight at prohibitive review cost. The framework also borrows thin slicing - rapid judgment from a narrow informational sample - applying it to domain-expert "golden examples," which carry genuine authority but also the parochial bias of the domain from which they originate.

3. Core Analysis

3.1 Shared Domain and Plan Representation

What a model "knows" about a term is not necessarily what the user means; definitions diverge according to personal or organizational context. The harness is designed to force clarifying back-and-forth before planning begins, surfacing hidden assumptions rather than permitting the agent to guess and proceed - explicitly avoiding "yes-man mode" on vague requests. Mechanically, this is implemented as a plan graph: planning is decomposed into discrete nodes or decision points rather than a single linear chat, so that individual elements can be revisited without destabilizing the surrounding conversation. Each node is annotated to indicate whether its content originated from the user, was inferred by the model, or remains an open question - making the shared-domain principle inspectable rather than implicit.

3.2 Consistency Through Deterministic Code

A recurring failure mode is that language models asked to perform arithmetic or financial calculations directly within a chat produce inconsistent, non-deterministic results across runs, even on identical categorization tasks. The diagnosis offered is structural: "you're asking the stochastic machine to produce a consistent output, so you're likely not going to get it." Instructing the model to "do the right thing" does not resolve this, because instructing a non-deterministic system still yields non-deterministic output.

The fix is architectural rather than prompt-based: once a calculation is identified during planning, it is implemented as versioned, tested code, and the agent calls that code as a tool rather than reasoning through the math as generated tokens. Versioning matters independently - if the underlying code for a calculation changes, the harness surfaces that fact so users know a previously trusted result may no longer be comparable to a new one. Tests additionally function as guardrails: an attempt to sabotage a calculation (for example, substituting aggregation for multiplication) triggers test failures that block the change from propagating.

3.3 Evidence and Provenance

Users must be able to trace conclusions back to their source - "show the receipts" - to substantiate financial decisions. The harness treats information sources as non-equivalent: unstructured chat content is labeled interpretation, agent reasoning or memory recall is labeled assumption, and submitted documents such as bank statements are labeled fact. This labeling is surfaced directly in the interface; clicking any underlined figure reveals its full provenance chain, including whether it was user-provided, agent-inferred, or still open.

The motivating failure case is instructive: an agent fixated on a single example value supplied mid-conversation and propagated it incorrectly throughout a presentation, with no flagged evidence trail to intercept the error before it reached senior leadership. The parallel drawn is to "master file" spreadsheets whose underlying assumptions change silently, producing wrong downstream results for users unaware of the change - a risk the presenters argue applies equally to agent-generated code and data pipelines absent traceability.

3.4 Accountability and Verification

Because "agents face no real-world consequences, you do," humans must retain oversight mechanisms, but the design goal is "few gates with very high signal" rather than either exhaustive line-by-line review or unchecked trust. Review is directed at judgment calls and disambiguation points rather than trivial decisions, analogous to reviewing task-level outcomes rather than every diff line. The plan graph supports branching conversations that merge back into the main plan, enabling correction of a single misplaced input - such as one mis-dictated word - without resetting the entire analysis.

Verification supplies a second, independent reference point to reduce blind spots without displacing the original decision-maker's accountability. Because expert golden examples can themselves be thin-sliced and domain-biased - illustrated by a case where a stakeholder from a different team overturned an analyst's seemingly correct SQL result using independent domain knowledge - the demonstrated harness includes a dedicated "challenge" agent branch that reviews plan assumptions and feeds proposed changes back into the main plan, in contrast to a static LLM-as-judge that produces a verdict with no feedback loop.

4. Technical Insights

Implementation details reinforce the five principles. State persists in a local directory acting as the plan's source of truth, deletable to reset to a fresh example - a lightweight, inspectable persistence model rather than opaque session state. The plan itself renders as a clickable graph UI, with each node exposing its origin and evidence trail on demand. Financial calculations (for example, per-guest cost or savings schedules) are implemented as separate, testable scripts invoked via tool calling, isolating deterministic computation from free-form generation. Branching sub-conversations - tangential questions or the verification agent itself - can either merge into the main plan or be dismissed without contaminating the primary context, limiting blast radius from exploratory detours.

A key trade-off is scope: a dedicated harness is warranted when reproducibility matters, stakes are high, or a generic agent repeatedly fails on the same pitfall - not for every new use case, since excessive harness-building becomes "sharpening the axe" instead of using it. Low-stakes tasks, such as AI-generated social content, do not require this evidence, verification, and accountability machinery. At scale, prompt-only "skills" approaches with many user-level overrides become unpredictable, motivating deterministic backend code to cover edge cases reliably.

5. Discussion

The framework implies that trust is not a single property satisfied by one mechanism but a composite the harness must support through multiple, parallel surfaces - some users want raw calculations visible, others are satisfied by verification signals, and no single affordance serves every trust bar simultaneously. This has organizational implications: framing adoption around measurable business outcomes, such as reduced back-and-forth or fewer support tickets, manages leadership expectations more effectively than appeals to abstract trust in "the AI."

An open question concerns generalizability beyond finance: the five principles (shared domain, consistency, evidence, accountability, verification) appear domain-agnostic in structure, but their specific implementations - bank-statement-as-fact, per-guest-cost scripts - are finance-specific. Future work could examine how the plan-graph and challenge-agent patterns transfer to other high-stakes domains such as healthcare or legal analysis, where evidentiary standards and accountability structures differ materially from banking.

6. Conclusion

The central contribution is a shift in framing: as agentic systems close the speed gap in financial analysis, the design problem becomes building scaffolding that earns trust by construction rather than by capability alone. The five principles translate into concrete mechanisms - provenance-annotated plan graphs, deterministic versioned calculation tools, evidence labeling, gated accountability checkpoints, and an adversarial challenge loop - that collectively replace reliance on agent judgment with inspectable, correctable structure. Practically, teams building agentic financial tools should reserve this level of harness investment for genuinely high-stakes, reproducibility-sensitive workflows, and should evaluate adoption against measurable outcomes rather than treating trust as a property to be assumed from model capability.

Sources

• Stop Chatting With Spreadsheets - Francesco Galletta & Felippe Felisola Caso, Monzo - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub