Why No One Agrees on What Their AI Tokens Cost with Pratyusha Duvvuri, Cisco

Tokconomics - the observability and interpretability of AI token usage and cost - has become a critical, underserved need because organizations now face custom

By Sean Weldon

Why No One Agrees on What Their AI Tokens Cost: Tokconomics as an Emergent Operational Discipline

Abstract

This synthesis examines tokconomics - defined as "the observability and interpretability of token usage and cost" - as an emergent operational discipline within organizations deploying large language models (LLMs) at scale. Drawing on customer discovery across approximately 100 companies and the development of a general-availability (GA) observability product, the analysis identifies three concurrent shifts driving demand: a status-driven "token maxing" phase, a subsequent pivot toward cost efficiency, and the proliferation of bespoke, non-public pricing agreements across model providers. The work establishes a three-segment stakeholder model (engineers, engineering managers, finance) and demonstrates that no single-provider dashboard satisfies all three. Principal technical contributions include a dual-hierarchy attribution model separating organizational reporting structure from financial cost structure, a data-minimizing ingestion design, and role-based access control. Implications extend to any organization attempting to justify AI expenditure to executive leadership.

1. Introduction

The rapid operationalization of LLMs has outpaced the financial instrumentation required to govern it. Where traditional cloud spend benefits from a mature ecosystem of cost-management tooling, token-denominated consumption remains largely opaque at the organizational level. One characterization offered during customer discovery captures the problem concisely:

"Token usage is you're driving a taxi with a meter and you have no clue what the taxi is going to show you at the end."

This absence of forward visibility distinguishes token consumption from prepaid or capacity-reserved models, where cost is bounded ex ante.

The central thesis advanced here is that tokconomics has become a critical and underserved organizational need. Three conditions converge to produce this gap: (i) model selection has become genuinely use-case dependent rather than uniformly maximal; (ii) provider pricing is increasingly negotiated per customer and therefore not inferable from public rate cards; and (iii) no existing tooling maps aggregate spend simultaneously onto organizational and financial hierarchies for stakeholders with materially different questions.

This analysis proceeds by establishing the economic backdrop (Section 2), characterizing the stakeholder segmentation and attribution problem (Section 3), presenting implementation findings (Section 4), and discussing open problems (Sections 5-6).

2. Background and Related Work

Three sequential trends in enterprise AI economics provide the intellectual context. The token-maxing phase treated token expenditure as a proxy for AI adoption maturity: Jensen Huang is reported to have remarked that a 500,000-employee company not spending at least approximately $250,000 on tokens should be cause for alarm, under the assumption that high consumption signals competitive seriousness. This posture proved unstable. A publicized instance in which Ramp's chief executive disclosed $1.5 million in token spend within a single week generated significant concern despite the prevailing token-maxing sentiment, marking an inflection toward cost scrutiny.

Prior art consists principally of first-party provider consoles - the OpenAI portal, the Anthropic dashboard, GitHub Copilot reporting. These are single-provider by construction, do not unify cross-provider consumption, and do not segment by organizational role, leaving a structural gap that the tokconomics framing seeks to name and address.

3. Core Analysis

3.1 Use-Case-Dependent Model Selection

Efficiency pressure has produced differentiated model routing rather than uniform maximization. One reported evaluation found a cheaper model, Luna, scoring 96% against a frontier alternative, Astra/GPT6, at 99%. The resulting decision rule is instructive: internal tooling absorbs the three-percentage-point quality deficit in exchange for lower unit cost, whereas external, customer-facing surfaces justify paying for the marginal quality gain. This indicates that cost governance in practice is not a blanket reduction mandate but a routing problem contingent on the consequences of quality variance in a given surface.

3.2 Custom Pricing as a Structural Driver

Every major model provider now offers non-standard, per-customer pricing agreements - for example, some customers paying 3-4 cents per credit while others pay nothing up to a $30 threshold and 10 cents per credit thereafter. Because these agreements are privately negotiated, public rate cards cannot be used to infer actual spend, and organizations cannot benchmark cost externally without internal instrumentation. This structural opacity is identified as a primary reason tokconomics has become organizationally necessary rather than merely convenient.

3.3 The Three-ICP Stakeholder Model

A survey of approximately 100 companies found no consistent single definition of tokconomics across roles, motivating a three-segment ideal customer profile (ICP) model: engineers (budget tracking, model/skill selection), engineering managers (cross-team usage, adoption-versus-cost tradeoffs), and finance (budgets, alerting, ROI justification to executive leadership and the board). Each stakeholder asks a structurally different question of the same underlying usage data, and no existing single-provider dashboard serves all three simultaneously.

3.4 Dual-Hierarchy Attribution

A central technical finding is that organizational reporting hierarchy (sourced from systems such as Workday or ADP) and financial cost hierarchy (a COGS/OPEX-based department tree) are structurally distinct even when they map to the same underlying individuals. The product therefore maintains two parallel attribution trees: an organizational rollup from the chief executive down through direct reports to individual contributors, and a separate finance-facing map terminating in departments, financial controllers, and analysts. A real customer case illustrates the value of this slicing: filtering by job family revealed that a sales team exhibited higher token usage and cost than the engineering team, a finding not discoverable through either hierarchy alone.

4. Technical Insights

Implementation required three categories of data ingestion, each constrained by sensitivity and reconciliation considerations.

5. Discussion

The findings suggest that cost observability for LLM consumption cannot be solved by extending existing cloud-cost tooling, because the unit of attribution (tokens, credits, thinking-pattern tiers) and the pricing structure (customer-specific, retroactively correctable) differ materially from metered infrastructure spend. The dual-hierarchy design - separating organizational reporting from financial cost structure - is likely generalizable beyond this specific product, since any organization reconciling engineering-reported adoption metrics with finance-reported budget variance faces the same structural mismatch.

A notable open problem is that tokconomics as a domain lacks a fixed "destination": the problem itself evolves as provider pricing models, model routing strategies, and organizational AI maturity continue to shift. This implies that any tooling built for this space must be architected for schema and hierarchy change rather than a static reporting model. Reconciliation of retroactively revised provider billing further compounds this, since personnel and organizational changes (departures, management reassignments) complicate long-term, historically accurate attribution.

The planned extension toward "ROI inference" - tying token spend to business outcomes such as customers landed or migrations completed - represents a logical next step but is not yet validated in the source material, and remains an area where causal attribution between token spend and business outcomes is likely to be methodologically difficult.

6. Conclusion

This analysis establishes tokconomics as a necessary response to three converging conditions: use-case-dependent model selection, proliferating custom provider pricing, and the absence of unified, role-differentiated cost visibility. Its principal technical contribution is a dual-hierarchy attribution model that reconciles organizational and financial views of the same spend, combined with data-minimizing ingestion and strict RBAC.

For practitioners, the practical takeaway is that cost governance for LLM deployment should be designed around the distinct questions asked by engineers, engineering managers, and finance, rather than a single aggregate dashboard. Organizations evaluating AI spend tooling should specifically assess whether a given solution can unify cross-provider usage, reconcile retroactive billing corrections, and map cost onto both reporting and financial hierarchies independently.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub

Sean Weldon builds apps and custom software at 22Atlas.

Start your build

More about Sean