Insurance Claims Are AI's Messiest Data Problem - Denys Linkov, Wisedocs

Insurance claims data, especially in complex medical claims processing, can power enterprise-wide AI transformation - but only after solving foundational data ...

Insurance Claims Are AI's Messiest Data Problem

Abstract

Insurance claims processing - particularly in medically complex lines such as workers' compensation and liability - represents a high-value but underexploited domain for applied artificial intelligence (AI). This synthesis, drawing on operational experience from Wisedocs and the MLCR benchmark developed with Artificial Analysis, argues that claims data can anchor enterprise-wide AI transformation only after foundational problems of digitization, deduplication, and governance are resolved. Evidence indicates that paper-based workflows, non-standardized intake forms, and duplicate record rates reaching 50% or higher constitute the binding constraint on AI value - not model capability. A layered extraction pipeline combined with domain-specific benchmarking demonstrates measurable progress: accuracy on complex claims-reasoning questions rose from 20% to 75% within months. Practical implications include architectural guidance on preferring extractive models over large language models (LLMs) for certain tasks, and how human-in-the-loop design addresses the majority of regulatory concerns.

1. Introduction

The Property & Casualty (P&C) insurance market - encompassing automobile, general liability, and workers' compensation lines - represents approximately $1 trillion in annual premiums. Profitability is governed principally by the loss ratio, the proportion of collected premiums disbursed as claim payouts, which typically hovers near 50%. Marginal improvements in claims adjudication speed and accuracy therefore translate directly into underwriting margin and, downstream, consumer pricing.

Claims adjudication nonetheless remains slow: weeks in favorable cases, years in litigated ones. The central thesis advanced here is that this latency is primarily a data problem rather than a model problem. AI should be selected as a solution only where it addresses a defined business objective - improved patient outcomes, faster claim resolution, or reduced leakage - rather than adopted as a hype-driven overlay. As one practitioner framing puts it: "It's really important again to think about the problem that we're trying to solve and then only choose AI if it's the right solution."

This analysis proceeds by establishing the state of digital transformation in healthcare and insurance (§2), analyzing data quality pathologies and the evaluation methodology developed to measure AI performance against them (§3), presenting pipeline architecture and model selection guidance (§4), and discussing regulatory and organizational implications (§5).

2. Background and Related Work

Claims processing sits at the intersection of two incompletely digitized industries. Electronic health record (EHR) adoption stood at approximately 8% in 2008 and reached only 80-90% following the COVID-19 pandemic - a seventeen-year diffusion curve accelerated by exogenous shock rather than organic incentive. Concurrently, the global fax market exceeds $7 billion and continues to grow, reflecting persistent paper-based transmission across physician offices, legal departments, and carriers.

Structural fragmentation compounds this. Workers' compensation intake forms differ across all 50 U.S. states with no national standardization, and electronic medical records (EMRs) are inconsistently structured across hospitals and provider networks even within a single jurisdiction. Regulatory oversight is similarly distributed: the National Association of Insurance Commissioners (NAIC) publishes guidance and the canonical line-of-business taxonomy (Property & Casualty, Life, Accident & Health), while individual states set binding operational constraints, including claims turnaround windows ranging from hours in Texas to 21 days elsewhere.

3. Core Analysis

3.1 Data Quality Pathologies in Claims Records

Claims data exhibits structural defects that precede any question of model performance. Duplicate records can constitute 50% or higher of claims data in some jurisdictions, inflating processing volume without adding information. Commingled patient records - for example, conflating "Bob Joe" with "Bob Joe Jr." - create HIPAA violations and direct regulatory exposure. File sizes present a separate engineering challenge: one documented claim file reached 8GB, combining images, x-rays, ultrasounds, and text documents in a single PDF, exceeding the practical limits of both conventional viewers and AI context windows.

Record inconsistency extends to content quality. Conflicting diagnoses across providers and missing referenced materials - such as x-rays cited but not included in a claimant package - are common and difficult to track manually. This is compounded by caseload pressure: claims adjusters managing approximately 200 active claims cannot feasibly track every outstanding follow-up, creating systemic blind spots that compound over the multi-week-to-multi-year adjudication timeline.

3.2 Benchmarking AI Performance on Claims Reasoning

To quantify AI readiness against this messy substrate, Wisedocs developed the MLCR (Multi-Level Claims Reasoning) benchmark in partnership with Artificial Analysis. The benchmark evaluates models across six tiers of question difficulty against human-expert validated answers, scored along three axes: conciseness, accuracy, and completeness.

Initial results showed highest accuracy on complex reasoning questions at only 20%, rising to 75% within months as reasoning models improved - a trajectory indicating rapid underlying model progress once properly measured. Notably, model families exhibited distinct failure modes: GPT models showed higher raw accuracy but tended to omit key information, while Anthropic models were more complete but slightly less accurate. This divergence motivated a triple-judge verification system with calibrated judges, rather than reliance on a single evaluation model, to validate outputs reliably. Despite performance gains, a human remains in the loop at every stage, reflecting both regulatory necessity and residual error risk.

3.3 Cross-Claim Analytics and Portfolio-Level Insight

Once individual claims are extracted and structured, the value proposition shifts from single-claim processing to portfolio-level analytics. Patient travel patterns within provider networks become analyzable, as do relational networks linking lawyers, doctors, chiropractors, and physiotherapists who recur together across claims. Co-occurrence analysis of medical conditions, using ICD codes, reveals diagnostic patterns across providers, while cost variation analysis across doctors and treatments identifies drivers of premium inflation. Year-over-year cost trend analysis - for instance, a 10% increase in a given treatment's cost - enables counterfactual modeling of network changes. One cited case tied poor claims data processing to $1 billion in additional overbilling, underscoring that data quality failures scale directly into financial loss.

4. Technical Insights

The extraction pipeline underlying this analysis follows a staged architecture: OCR and image processing extract raw text from scanned or photographed records; NLP extraction models, including vision-language models (VLMs), structure the resulting unstructured data; classifiers and a summarization layer sit atop this structured output; and a human verification step precedes any higher-value analytical layer.

A key implementation decision is model selection by task. Classification and extractive models are preferred over general-purpose LLMs for many sub-tasks due to cost and accuracy/hallucination concerns - a direct application of the principle that "LLMs are not the right solution for everything." Wisedocs trains proprietary models rather than relying solely on general-purpose LLMs, citing both cost efficiency and accuracy control as justification. This reflects a broader trade-off: general-purpose LLMs offer flexibility and reasoning capability (as reflected in MLCR gains), but narrow, trained classifiers offer predictability and auditability where outputs feed into regulated decision processes. Human-in-the-loop verification is positioned as addressing roughly 70% of regulatory concerns, functioning as a risk-mitigation layer independent of model architecture.

5. Discussion

The evidence suggests that AI capability is no longer the primary bottleneck in claims processing; data infrastructure and organizational readiness are. The persistence of a $7 billion fax market and non-standardized state-level intake forms indicates that digitization itself remains unfinished business, meaning AI deployments are frequently built atop incomplete data foundations. This has a direct fraud and efficiency implication: inability to process claims efficiently functions analogously to an under-defended system vulnerable to volume-based exploitation, as "if your website is easy to DDoS."

A further asymmetry worth noting is adoption incentives: plaintiff-side legal practices, compensated via contingency fees ranging from 9% to 40% depending on jurisdiction, have adopted AI faster than defense-side carriers, suggesting that AI diffusion in claims follows incentive alignment rather than technical availability alone. This raises an open question for future investigation: whether carrier-side adoption will accelerate once governance frameworks - audit trails, human-in-the-loop checkpoints - are standardized, or whether defense-side incentive misalignment will persist as a structural drag on adoption regardless of technical maturity.

6. Conclusion

This synthesis demonstrates that claims data, despite its structural messiness, can support meaningful AI-driven transformation when approached through a disciplined pipeline: extraction, verification, and only then analysis. The rise in MLCR benchmark accuracy from 20% to 75% confirms that model capability is advancing quickly, but this progress is contingent on upstream data cleanliness and human oversight rather than a substitute for it. Practically, organizations pursuing AI in claims processing should prioritize deduplication, standardized extraction protocols, and governance frameworks before layering on advanced reasoning models, ensuring that AI serves clearly defined objectives - patient outcomes and claims efficiency - rather than functioning as an unexamined technological overlay.

Sources

• Insurance Claims Are AI's Messiest Data Problem - Denys Linkov, Wisedocs - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub