'Mousepower: agents that can’t be measured, can’t be managed. - Maximillian Piras, Yutori'

'Agents have a measurement problem: without clear metrics tying token spend to verifiable value/outcomes, customers cannot calculate ROI, which stalls adoptio...'

By Sean Weldon

Mousepower: Agents That Can't Be Measured, Can't Be Managed

Abstract

Autonomous software agents are being deployed at increasing scale, yet their economic value remains poorly quantified. This synthesis argues that agents face a measurement problem: in the absence of metrics connecting token expenditure to verifiable outcomes, prospective adopters cannot compute return on investment (ROI), and adoption stalls. The analysis draws a historical parallel to James Watt's invention of horsepower, an imprecise but communicatively powerful metric that allowed buyers to calibrate the value of steam engines against the horse gins they replaced. Contemporary failure modes are characterized, including a cyclical "doom loop" of token overspending followed by austerity, and the migration of the engineering bottleneck from code generation to code review. Two conceptual contributions are presented: Mouse Power, a principle requiring agent vendors to supply verification rubrics, and an uncertainty matrix identifying NP-style tasks as the optimal domain for agent deployment.

1. Introduction

Agentic systems - large language model (LLM)-driven processes that decompose goals into multi-step action sequences - have moved rapidly from demonstration to production. A subclass of particular interest comprises computer-use models: agents that operate graphical interfaces in the manner of a human operator, applicable where no Application Programming Interface (API) or Model Context Protocol (MCP) integration exists. Such systems extend automation to software that was never designed for programmatic control.

Adoption patterns reveal a persistent gap between capability and confidence. Practitioners report backgrounding multiple agents in parallel to explore alternative design directions while attending to a primary task. The subjective experience is one of high productivity; the arrival of the invoice frequently reverses that judgment. Customers express enthusiasm while simultaneously reporting that they are "just scratching the surface," lacking intuition about which use cases justify the spend. Notably, this analysis cautions that early adopters observed in technically sophisticated settings are unrepresentative of the mainstream market - an Upton Sinclair-style bias in which enthusiasm is concentrated among those already positioned to benefit - while many prospective users have not moved beyond copy-pasting into chat interfaces.

The central thesis advanced here is that this is not primarily a capability problem but a measurement problem. Token consumption is observable and immediate; value is diffuse and delayed. Without a shared unit of account linking the two, buyers cannot reason about ROI, and adoption reduces to intuition or fear of missing out (FOMO). This paper establishes the historical precedent for solving such problems (§2), diagnoses current industry failure modes (§3), formalizes a task-selection framework (§4), and considers implications for vendors and practitioners (§5).

2. Background and Related Work

When James Watt sought to commercialize the steam engine, the incumbent technology was the horse gin, a rotary power source driven by harnessed horses. Buyers possessed no vocabulary for comparing the two technologies. Watt's response was empirical but informal: he studied horse gins in operation and derived an approximate figure - horsepower - expressing engine output in units of the familiar alternative. The metric was not scientifically rigorous, nor accurate in any strict sense; its function was communicative, permitting buyers to perform an arithmetic comparison and arrive at a defensible purchasing decision. Watt also had to overcome non-economic resistance, since "it's hard to beat the vibes of horses"; a tangible calculation provided the counterweight that sentiment alone could not supply.

Claude Shannon's information theory supplies the second theoretical pillar. Entropy quantifies uncertainty within a probability distribution, and cross-entropy is already central to the training objectives of the models underlying modern agents. This analysis extends that concept metaphorically to task selection, treating uncertainty in task execution and uncertainty in outcome verification as the two dimensions that determine whether an agent is an appropriate solution to a given problem.

3. Core Analysis

3.1 The Doom Loop of Token Spend

A recurring failure mode, borrowed conceptually from the fintech firm Ramp, is described as a doom loop: teams maximize token usage, encounter unexpectedly high bills, retreat into austerity, and disengage from agent tooling until FOMO reignites usage. This cycle is illustrated by a public example from Coinbase's CEO, who reported that shifting default model usage toward cheaper models - reserving frontier models for harder tasks - caused AI spend to diverge from token volume growth. The implication is that token count alone is a poor proxy for cost or value; spend and usage can move in opposite directions depending on model allocation decisions, undermining any measurement approach anchored solely to raw token counts.

3.2 From Code Generation to Code Review

Even within highly AI-adopting engineering organizations, the operational bottleneck has migrated. Coding agents now generate pull requests faster than human reviewers can verify them, producing a condition summarized as "we're all dying by a thousand pull requests." Anthropic is credited with having substantially addressed code generation, but not code review; verification, not production, has become the binding constraint. This shift suggests that assumptions underlying existing code review practices - designed for human-paced contribution - require revisiting for an agentic context, a position attributed to Noah Hein. The broader lesson is that token-focused measurement misses the actual point of friction: outputs must be traced to clean, verifiable outcomes such as bugs squashed or support tickets closed, not merely tokens consumed.

3.3 Mouse Power as Principle

An initial, deliberately playful attempt to construct a literal analog to horsepower involved measuring mouse-movement efficiency using a Claude-vibe-coded device. This effort was judged a "fool's errand," given the high-dimensional information space involved in characterizing agent behavior through physical proxy measurements. The resulting conclusion is that Mouse Power should not be understood as a literal metric but as a governing principle: any vendor selling agentic capability has an obligation to supply customers with a rubric by which agent output can be verified. This reframes the measurement problem from "how many tokens were used" to "how was the result validated," shifting the locus of trust from consumption to verification.

4. Technical Insights

The uncertainty matrix formalizes task selection along two axes: uncertainty in task steps (x-axis) and uncertainty in acceptance or verification criteria (y-axis).

A key limitation is that this framework is qualitative rather than quantitative; unlike horsepower, no numerical unit is proposed. Its value lies in providing a heuristic filter for identifying viable agent use cases prior to investment, rather than a post hoc accounting method.

5. Discussion

These findings suggest that the agent industry's measurement gap is less a data-availability problem than a conceptual one. Token counts are abundant and precise, yet precision at the wrong level of abstraction does not resolve buyer uncertainty about value. The Watt precedent indicates that imperfect, communicatively effective metrics can be sufficient to unlock adoption, provided they connect a novel technology's output to a unit the buyer already understands. For agents, that unit is not computation but verified outcomes.

The migration of bottlenecks from generation to verification - visible in the code review example - suggests that future infrastructure investment should prioritize verification tooling over further gains in generation throughput. This has direct implications for vendor incentives: suppliers of agentic systems that fail to also supply verification rubrics may be externalizing the hardest part of the problem onto customers, perpetuating the doom loop described in §3.1.

6. Conclusion

This synthesis contributes two conceptual tools to the problem of agent ROI measurement: the Mouse Power principle, which obligates vendors to pair agent capability with verification rubrics, and the uncertainty matrix, which identifies NP-style tasks - easier to verify than to execute - as the optimal deployment target for agentic systems. Practically, organizations evaluating agent adoption should assess candidate tasks along both uncertainty axes before investment, and vendors should treat verification tooling as a core deliverable rather than an afterthought. Future work should explore whether quantifiable analogs to horsepower can be derived for specific task categories, narrowing the gap between this qualitative framework and an operational metric.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub