How We Mapped a 35-Million-Line Mainframe in 4 Days with Stephen Chin, Neo4j & Neeraj Deshmukh, Blitzy

Legacy mainframe modernization fails when AI only has local code context; by building a knowledge graph of the entire software estate (code, data, business r

Mapping the Dark Room: Knowledge Graphs as the Missing Layer in AI-Driven Mainframe Modernization

Abstract

Legacy mainframe modernization has historically failed not because AI cannot generate correct code, but because code-generating agents operate with local rather than system-level context. This synthesis examines an architectural response to that limitation: constructing a continuously updated knowledge graph of an entire software estate - programs, batch jobs, copybooks, data contracts, business rules, and runtime behavior - and using it to supply agents with precise, multi-hop context packets. Evidence from the Blitzz platform indicates that graph-grounded agent orchestration, combined with mandatory human approval gates and validation compute exceeding generation compute, can achieve byte-for-byte behavioral parity on production workloads. Reported results include a 35-million-line mainframe mapped in four days, a tier-one bank monolith decomposed in 18 days against an 18-engineer-week plan, and 85% autonomous code completion on a 25-million-line platform. The findings suggest retrieval topology, not context window size, is the binding constraint on enterprise code transformation.

1. Introduction

Critical financial infrastructure in major financial centers continues to run on mainframe systems whose business rules, nightly batch jobs, and data contracts have accreted over decades. These systems are frequently described as untouchable: institutional knowledge of their behavior has eroded, while the cost of behavioral regression is measured in regulatory exposure and direct monetary loss. Traditional mainframe-to-Java migration programs have consequently been multi-year, bug-prone, under-tested, and frequently abandoned before completion.

The arrival of capable code-generating LLMs appeared to offer relief, but early results exposed a specific and instructive failure mode. The central thesis examined here is that modernization efforts fail when AI possesses only local code context, and that the corrective is a knowledge graph of the complete software estate - code, data, business rules, and runtime behavior - against which agents can be planned, constrained, and validated. A knowledge graph, in this context, is a structured representation of software artifacts and their relationships (programs, jobs, copybooks, outputs) that preserves provenance and supports multi-hop traversal, as distinct from flat document or vector stores.

This analysis proceeds by establishing the failure mode (§2), comparing retrieval topologies and the empirical limits of context windows (§3.1-3.2), describing graph architecture and orchestration (§3.3-3.4), examining how behavioral parity is proven (§3.5), and reviewing reported outcomes and their implications (§4-§6).

2. Background and Related Work

The failure mode is illustrated by a concrete instrument: the AWS Card Demo reference application. Its interest calculation is not a standalone formula but a distributed behavior spanning a batch job, an interest program, copybooks defining record layouts, and VSAM data output. Instructing an agent to "convert this formula from COBOL to Java" retrieves only the formula. The resulting translation omits record precision, account updates, transaction output, and the batch contract that downstream jobs depend upon.

As the source material frames it: "The failure here is not that AI can't write the code. It's that AI doesn't know the 14 other places where that logic, that code is being used." This reframes modernization as a knowledge-retrieval problem rather than a generation problem. Prior approaches to grounding LLMs in large codebases - vector similarity search, relational metadata repositories, and MCP (Model Context Protocol) servers - each impose topological limits examined below. Related tooling referenced includes Blu Age sandboxes for transpilation validation and conventional IDE-based agentic assistants as a performance baseline.

3. Core Analysis

3.1 Context Blindness as a Topological Constraint

Retrieval mechanisms differ not merely in capacity but in the shape of queries they can answer. Vector databases support similarity search only, surfacing code that resembles a query target rather than code that depends on it. Relational databases and MCP servers extend repository size but remain limited to single-hop lookups via SQL-style queries - they can answer "what is this," not "what else touches this, and what touches that." Graphs, by contrast, support multi-hop traversal: a query can follow a chain from formula to program to batch job to copybook to downstream consumer in a single structured path. This distinction - not raw storage volume - is presented as the determining factor in whether an agent receives accurate context.

3.2 The Illusion of Large Context Windows

A second constraint concerns context windows advertised at the million-token scale. Reported usable context is roughly 200,000 tokens or fewer - approximately 20-30% of advertised capacity - equating to roughly 200,000-300,000 lines of code before information degrades. Accuracy diminishes as a context window fills, meaning volume alone does not guarantee retrieval fidelity. The underlying problem is structural: "Context is serial, but information has relationships. Information has connections between it." Serial token sequences cannot natively encode the dependency graph that governs whether a code change is safe.

3.3 Knowledge Graph Architecture

Blitzz constructs a graph in which programs, jobs, copybooks, outputs, and their dependencies are explicit nodes and edges. Each edge retains provenance: source repository location, revision, symbols touched, and the evidence (file or source span) from which the relationship was extracted. The graph is incrementally updated on every code merge, invalidating stale information so agents never act on outdated structure. This is described as enabling "infinite code context" - not memorization of an entire codebase, but access to the right slice of information at the right time. Operationally, each agent receives a task context packet containing a task description, a relevant graph slice, constraints, available evidence, and open questions. The platform is characterized as functioning "as a compiler for the entire knowledge in the enterprise," reverse-engineering the estate and converting requirements into an agent action plan.

3.4 Human-in-the-Loop Orchestration and Validation Controls

Human approval is required at every key decision point before agents act on a plan. After approval, code generation proceeds with validation embedded at each step, producing a pull request alongside evidence documentation. Orchestration operates at scale - thousands of agents share a single plan without context loss between them - and validation compute is reported to exceed generation compute. Four controls are identified as underpinning every change: system-level context (the knowledge graph), continuous validation (testing), explicit guardrails (security, governance, compliance), and human-in-the-lead decision ownership.

3.5 Proving Behavioral Parity

The elements preserved across modernization include monetary precision, rounding behavior, state, balances, transaction records, failure modes, and operations. Parity is established through equivalence testing comparing baseline and modernized systems across inputs, outputs, returned values, persisted state, transaction output, and error outcomes - not merely happy-path scenarios. Testing layers span unit tests on generated code, compiled build checks, runtime integration checks, data parity validation on every write, and identical end-to-end workflow verification. A representative failure case: Java's default rounding behavior diverges from COBOL's, producing a one-cent discrepancy per transaction that is undetectable without parity testing but compounds materially across millions of transactions.

4. Technical Insights

Several implementation-relevant findings emerge. First, retrieval topology dominates context volume: multi-hop graph traversal answers dependency questions that similarity search and single-hop lookup cannot, regardless of how much text a model can ingest. Second, provenance metadata on graph edges (repository location, revision, extracted evidence) is what allows an agent-generated plan to be audited and trusted rather than taken on faith. Third, incremental graph updates on every merge are necessary to prevent agents from acting on stale dependency information, which is a non-trivial engineering requirement for live production systems. Fourth, the allocation of more compute to testing than to generation is a deliberate trade-off: for behavior-preserving transformation, correctness verification is the harder and more consequential problem. A limitation worth noting is that reported metrics (4-day mapping of 35 million lines, 18-day decomposition against an 18-engineer-week estimate) originate from the platform vendor and the associated case studies; independent replication data is not presented in the source material.

5. Discussion

The broader implication is that enterprise code transformation is best understood as a knowledge-representation problem before it is a code-generation problem. This reframes the competitive and research question in AI-assisted software engineering away from "which model generates the best code" and toward "which system supplies the most accurate, provenance-tracked, multi-hop context." It also suggests that context-window scaling, while useful, is not a substitute for structured retrieval - a point with implications beyond mainframe modernization, for any domain where code or documents carry dense cross-references.

A second implication concerns the durability of the knowledge layer once built. The source material frames Blitzz as a transformation platform rather than a one-time modernization tool: once governance rules, business-rule mappings, and test suites exist, subsequent work - new features, security remediation, regulatory compliance checks such as DORA, or fraud detection - becomes incrementally cheaper rather than starting from zero. This suggests that the economic case for building the graph should be evaluated against its reuse value, not solely against the first migration project.

Open questions remain regarding generalizability outside financial-services mainframe contexts, the cost and complexity of initial graph extraction for codebases with less standardized structure than COBOL/copybook systems, and how these results compare under independent, non-vendor-controlled benchmarking.

6. Conclusion

The central contribution of this analysis is the identification of retrieval topology - not model capability or context window size - as the binding constraint on reliable, large-scale code modernization. By encoding an entire software estate as a provenance-tracked, continuously updated knowledge graph, and by pairing agent action with mandatory human approval and validation compute exceeding generation compute, the approach reported here claims to achieve behavioral parity at a speed and scale unreachable by traditional human-led migration. The practical takeaway for technical teams is that investment in structured, multi-hop context retrieval - and in equivalence testing beyond happy-path coverage - is likely to yield more reliable AI-assisted transformation outcomes than investment in larger context windows alone.

Sources

• How We Mapped a 35-Million-Line Mainframe in 4 Days - Stephen Chin, Neo4j & Neeraj Deshmukh, Blitzy - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub