Make Your AI Agents Predictable With Graph Engineering with Zachary Blumenfeld, Neo4j

Graph engineering - designing agent work as explicit graphs of nodes and edges - becomes most powerful when graphs are stored as portable artifacts in a know

Make Your AI Agents Predictable With Graph Engineering

Abstract

This synthesis examines graph engineering, the practice of designing agent work as an explicit graph in which nodes perform work and edges route state between them. Positioned as a successor to prompt engineering and context engineering, it applies established computer science constructs - state machines, workflow graphs, and knowledge graphs - to agent orchestration. The analysis distinguishes three graph classes (execution, context, and procedural) and reviews research showing that compiling procedural knowledge into executable graphs increases task reward across heterogeneous domains. A loan-approval case study built on Neo4j's knowledge-layer architecture shows that compiled procedural graphs cut wall-clock latency substantially and permit smaller models to be substituted without loss of reliability. The central contribution is architectural: storing graphs as portable artifacts in a knowledge layer adjacent to data, rather than embedding them in a runtime, yields predictability, governance, auditability, and cost reduction simultaneously.

1. Introduction

Agent development has progressed through successive abstraction layers. Prompt engineering optimized single-turn instruction quality; context engineering optimized what information reaches the model's window. Graph engineering, a term popularized by a Peter Steinberger post in mid-2025, addresses a third concern: the structure of the work itself.

"Graph engineering is just the practice of designing agent work as an explicit graph, where nodes do work and edges effectively route state between them."

The underlying mechanisms are not novel. State machines, workflow graphs, and knowledge graphs are mature constructs within computer science. What is new is their systematic application to non-deterministic agent systems, in which a language model - rather than a compiler or scheduler - would otherwise decide control flow at every step. As Zachary Blumenfeld of Neo4j argues, this matters because "graph engineering lets agents really take on bigger, more complex work."

The central thesis advanced here is that graph engineering realizes its full value only when the graph is persisted as a portable artifact in a knowledge layer beside the data, rather than being locked inside a particular agent runtime. This paper defines the taxonomy of agent graphs (§2), examines the knowledge-layer architecture and related research (§3), analyzes a governed loan-approval implementation (§4), and discusses implications (§5).

2. Background and Related Work

Three distinct graph types operate at different layers of an agent system. Execution graphs are workflow graphs in which routers and agents act as nodes inside an agent harness, the familiar structures exposed by orchestration frameworks such as LangGraph and ADK. Context graphs represent decision traces and their connections to underlying domain data, binding what an agent concluded to the evidence that produced the conclusion - incorporating short-term memory (conversation history), long-term memory (entities extracted from source data), and reasoning/decision trees. Procedural graphs express skills and procedures as graphs, with typed steps including decision models, escalation points, and validation checks.

Execution graphs are comparatively well served by existing tooling. Context and procedural graphs remain underdeveloped and are the focus of this analysis, since they are where the portability argument chiefly applies.

3. Core Analysis

3.1 The Knowledge Layer Architecture

Neo4j's proposed architecture places a knowledge layer between systems of record (DataStax, BigQuery, Snowflake, documentation, markdown files) and agent runtimes. Standard protocols and artifacts live in this layer, making the output of a graph engineering process available to any runtime or agent rather than one specific framework. The product stack consists of a core graph database plus an "agentic brain," or memory tools layer, that hooks into the database to move context in and out. This separation is what enables portability: a graph compiled once can be consumed by LangGraph, ADK, or any other harness without re-derivation.

3.2 Compiling Procedural Knowledge into Graphs

A prior Neo4j research paper, published at a workshop, investigated whether procedural knowledge - past decisions or documented skills - could be compiled into a graph structure. The process takes written material, such as a skill markdown file together with its scripts, references, and assets, and converts it into nodes representing decisions, routing points, and executable scripts. Forcing an agent to run the compiled graph, rather than reason freely, increased task reward and performance across diverse fields including physics, computer science, security, and physical training. The experiments were originally run on Sonnet and later repeated on the smaller Haiku model, indicating that compilation benefits are not contingent on model scale.

An extension of this work, led by PM Will Lein, applies the same distillation technique to memory and context graphs: a skill is distilled directly from the context graph while remaining connected to the past decision traces and source data that produced it. This yields both a performance benefit and full provenance - the resulting skill is versioned, tracked, and auditable, supporting drift assessment and tool-call history review via the Neo4j Agent Memory Service (NAMS) reference architecture.

3.3 Customer Deployments

Two deployment patterns illustrate the practical pull toward graph-based governance. Consultancies working with asset managers use graphs to automate audit reports drawn from multi-source data, resolving entities that carry different ticker symbols or names across databases in a traceable, governed manner. Uber uses configuration graphs to manage geographically varying driver compliance regulations pulled from sprawling microservices, expressing regulatory logic alongside the underlying data so it can be operationalized consistently across regions.

4. Technical Insights

The loan-approval demonstration ("Rosa's Bakery") provides the clearest quantitative evidence for the architecture's value. The scenario combines a business-plan PDF, Databricks transaction data, lending policy data, and historical decisions in a single graph.

• Ungoverned baseline: initial agent-only runs produced decision chains of inconsistent length, and approve/deny/escalate decisions carried uncalibrated confidence scores - direct evidence of the predictability problem graph engineering targets.
• Compilation into a skill: successful runs were scraped and compiled into a graph-shaped skill stored as YAML, bundling scripts, assets, and source policy documentation for traceability. As Blumenfeld notes, "none of this - we're not creating a loop inside of our agent. All we're doing here is giving it a CLI to run this programmatic skill that we compiled from documentation or, in this case, from our memory graph."
• Workflow structure: execution proceeds through hard policy-rule checks, routing to a type-safe decision model for risk-tier and exposure assessment, an approve/deny outcome, report writing with a lint/check step, and a human-escalation router where needed.
• Latency and cost: a baseline agent-only run on Haiku/Fable took approximately 2.7 minutes (with other descriptions citing a ~7.5 minute baseline under different conditions). Executing the compiled procedure via the AIP library roughly halved run time; substituting Haiku for the compiled procedure further reduced wall-clock time to roughly 30 seconds to 2 minutes, with correspondingly lower cost.
• Governance affordance: runs are persisted and inspectable in the graph, tracking status, branches taken and not taken, and final approvals or denials, enabling post-hoc audit.

The implementation trade-off worth noting is that routing decisions move from the agent's discretion to programmatic control via the compiled CLI; this constrains flexibility in exchange for reproducibility, and Blumenfeld suggests a further refinement - "it would be great to even take some of this and send it to something a type-safe, Java type of model" - for the steps requiring strict correctness guarantees.

5. Discussion

The loan-approval results suggest a reframing of the model-selection question in agent system design: rather than choosing the most capable available model to compensate for weak task structure, practitioners can impose structure externally (via a compiled procedural graph) and substitute smaller, cheaper models with no apparent reliability loss. This has direct cost implications for production agent deployments operating at scale.

A second implication concerns portability as a design discipline rather than a convenience. By locating graphs in a knowledge layer adjacent to data stores, rather than inside a specific orchestration framework, organizations avoid re-encoding procedural and context graphs each time they adopt a new runtime. This is consistent with the broader industry trend toward separating agent "memory" and "procedure" from agent "execution engine."

Open questions remain regarding how context graphs should be maintained as source data drifts, how compiled skills should be re-validated after underlying policy changes, and how the type-safe step delegation Blumenfeld proposes (routing specific sub-tasks to deterministic, strongly-typed systems) should be formalized within the graph taxonomy. The research cited here spans a workshop paper and internal demonstrations rather than peer-reviewed, large-scale benchmarks, so claims about generalizability across domains warrant further validation.

6. Conclusion

Graph engineering reframes agent reliability as a structural problem rather than a purely model-capability problem: explicit nodes and edges, compiled from documented procedures and historical decision traces, measurably increase task reward, reduce latency, and permit smaller models to perform comparably to larger ones. The architectural contribution that makes this durable - storing graphs as portable YAML artifacts in a knowledge layer rather than embedding them in a runtime - decouples governance and auditability from any single orchestration framework. Practitioners building complex, high-stakes agent workflows should consider compiling procedural knowledge into inspectable graph artifacts early, both as a predictability measure and as a lever for reducing inference cost.

Sources

• Make Your AI Agents Predictable With Graph Engineering - Zachary Blumenfeld, Neo4j - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub