'AI on Your Lakehouse: Context Comes in Shapes, Not Queries - Zach Blumenfeld, Neo4j'

Graph-based semantic layers and document structure representations enable AI agents to effectively navigate both structured data warehouses and unstructured ...

By Sean Weldon

Graph-Based Semantic Architectures for Multi-Modal AI Agent Navigation

Abstract

This paper examines graph-based semantic architectures for enabling AI agents to navigate complex multi-modal data environments comprising both structured data warehouses and unstructured document collections. Traditional approaches - text-to-SQL for structured data and vector search for documents - exhibit critical limitations when scaled to hundreds of tables or millions of documents, frequently producing confident but incorrect responses. The research demonstrates that property graph representations of metadata, document hierarchies, and cross-document relationships enable deterministic navigation, semantic expansion, and global pattern detection. Through the Neo4j Carta framework and hierarchical document modeling, agents achieve reliable query execution across tactical and strategic information retrieval tasks. Key contributions include metadata-only semantic layers that avoid data migration, deterministic document loading with URI-based filtering, and community detection algorithms for identifying documentation gaps without LLM-based labeling.

1. Introduction

The proliferation of enterprise data across structured warehouses and unstructured document repositories presents fundamental challenges for AI agent systems. While text-to-SQL approaches enable natural language querying of relational databases and vector search facilitates semantic retrieval from document collections, both methods degrade significantly at scale. As observed in production deployments, "when those tables become massive and you get hundreds of tables or when you have large document stores where you have hundreds and thousands and millions of documents things can get lost and fall through the cracks." Agents operating at this scale frequently generate plausible but factually incorrect responses - a phenomenon particularly problematic in domains requiring high reliability such as automotive repair, healthcare, or regulatory compliance.

This research investigates graph-based semantic architectures as a solution to multi-modal data retrieval challenges. The central thesis posits that context for AI agents should be represented as shapes - structured graph patterns encoding relationships, hierarchies, and communities - rather than raw query results. The analysis draws from a practical implementation scenario involving AutoFix, a fictional auto repair chain managing structured work order data in BigQuery alongside unstructured technical manuals, safety bulletins, and recall notices accessed by three distinct personas: floor technicians requiring tactical guidance, leadership building co-pilot systems, and AI engineers developing agent capabilities.

The following sections examine three core graph-based patterns: the Connection Shape for structured data navigation, the Table of Contents Shape for hierarchical document traversal, and the Themes Shape for unsupervised pattern detection. Technical implementation details, strategic positioning considerations, and comparative performance characteristics are analyzed to establish practical guidelines for deployment in production environments.

2. Background and Related Work

Property graph data models represent information as nodes (entities), relationships (directed associations), and properties (attributes on both nodes and relationships). This formalism contrasts with relational models optimized for tabular data and vector embeddings designed for semantic similarity matching. Retrieval-Augmented Generation (RAG) architectures typically employ vector search to ground large language model responses in retrieved context, but struggle with queries requiring comprehensive coverage or negative evidence - what the research characterizes as "estate level questions" about global patterns and gaps.

Neo4j Carta represents a metadata-centric approach to semantic layer construction, creating graph representations of database schemas without extracting or transforming production data. As emphasized in the implementation philosophy, "we're not using the graph to copy the data over. There's not an ETL into graph. What we're doing is we're using the graph as a semantic layer." This strategy aligns with federated query architectures and addresses security and compliance concerns inherent in data migration approaches. The Graph Data Science (GDS) Library provides in-memory graph projections and algorithm execution frameworks, enabling community detection and centrality analysis at scale without requiring LLM-based classification.

The Leiden algorithm for community detection operates on graph topology alone, identifying clusters through modularity optimization controlled by a gamma parameter that adjusts resolution granularity. This deterministic approach contrasts with LLM-based document clustering methods that introduce latency, cost, and non-determinism into the analysis pipeline.

3. Core Analysis

3.1 Connection Shape: Metadata Graphs as Semantic Layers

The Connection Shape addresses the fundamental challenge of text-to-SQL generation at scale: understanding how to correctly join hundreds of tables across complex foreign key relationships. Neo4j Carta creates a metadata graph from warehouse schema information, establishing a hierarchy of database → schema → table → column nodes with explicit primary and foreign key relationships. This graph representation is constructed without ETL, maintaining data in the original warehouse while providing a navigable semantic layer.

The implementation utilizes an MCP (Model Context Protocol) server that allows Claude to read warehouse schema metadata and construct correct join paths. For example, when queried about which vehicles received specific parts, the agent traverses the metadata graph to identify the necessary joins between work orders, vehicles, and parts tables. The metadata graph optionally includes representative values from columns, enabling the agent to understand data distributions and cardinality without accessing production data directly.

This approach solves the critical problem of schema comprehension by providing pre-computed semantic relationships. Rather than requiring the agent to infer join logic from table and column names alone, the metadata graph explicitly encodes foreign key relationships as traversable edges. The deterministic nature of this representation ensures consistent query generation across repeated requests, avoiding the variability inherent in pure LLM-based SQL generation.

3.2 Table of Contents Shape: Hierarchical Document Navigation

The Table of Contents Shape provides deterministic navigation through unstructured document collections by creating a containment tree representing library → folders → documents → sections hierarchies. Each node receives a hierarchical URI (e.g., /technical_library/bulletins/tsb_recall_notice) that enables efficient filtering and scoping of search operations. Documents include cross-document links as named relationships, enabling navigation beyond linear search patterns.

The implementation employs parameterized Cypher queries with variable-length paths to allow agents to traverse the tree at specified depths or from specific URI locations. For example, the pattern (lib)-[:has*0..2]->(node) retrieves nodes up to two levels deep from a library root. This deterministic approach offers several advantages: it is idempotent across repeated loads, executes faster than LLM-based extraction, and works particularly well for inherently structured documents such as technical manuals with explicit section numbering.

Agents can drill down into document hierarchies and follow cross-document links without requiring semantic similarity matching. This capability proves essential for queries requiring comprehensive coverage, as it enables systematic traversal of entire document subtrees. The URI-based structure also facilitates post-filtering in full-text searches, allowing queries to be scoped to specific document collections (e.g., "search only within recalls folder").

3.3 Search Augmentation: Full-Text Indexing with Semantic Expansion

While hierarchical navigation provides deterministic traversal, many queries require keyword-driven discovery when the hierarchical path is unknown. The implementation creates a Lucene full-text index on document and section nodes, indexed on text properties. To address vocabulary mismatch between user queries and document terminology, the system employs semantic expansion: the model generates synonyms before query execution, transforming queries like "engine shuttering" into "misfire OR rough idle" for more comprehensive retrieval.

This approach complements outline navigation by enabling flexible keyword search while avoiding the computational overhead and semantic drift risks of vector embeddings. The URI-based filtering capability allows searches to be constrained to document subtrees, combining the precision of hierarchical scoping with the flexibility of full-text matching. As observed in implementation testing, this hybrid approach reduces the number of vector search hits needed for multi-step retrieval, thereby lowering hallucination risk in complex reasoning chains.

3.4 Themes Shape: Community Detection for Global Pattern Analysis

The Themes Shape addresses strategic, estate-level questions about documentation patterns, gaps, and relationships that cannot be answered through local search alone. The implementation applies the Leiden algorithm to detect natural clusters in the document link graph without LLM labeling - a purely structural analysis. A graph projection first collapses section-level relationships to document level: if section A links to section B, this is aggregated as document A linking to document B, producing a cleaner topology for community detection.

The algorithm output includes theme ID, a coherence metric (quantifying whether themes are tightly or loosely interlinked based on internal versus external edge density), top shared link types, and highest-centrality documents within each theme. The gamma parameter controls clustering granularity, with gamma=1 (default) producing coarser clusters and gamma=2 yielding finer-grained groupings. The hierarchical nature of the algorithm enables multi-level analysis of documentation structure.

This approach enables answers to questions that vector search cannot effectively address: "You can't really search for a negative. You have to search for things that are there. And so what I found is that graphs can be very useful when you start having these more global types of questions, these estate level questions." Examples include identifying documentation gaps (diagnostic codes with no documented procedures), finding unused documentation (procedures never referenced in work orders), and discovering unexpected document groupings that suggest organizational or taxonomic issues.

4. Technical Insights

The technical implementation reveals several critical design decisions and trade-offs. The Neo4j Carta MCP server ingests BigQuery metadata - tables, columns, primary and foreign keys - creating a property graph without copying production data. This metadata-only approach avoids security and compliance issues associated with data migration while providing sufficient semantic context for query generation. The Virtual Graph preview feature extends this concept by providing federated graph views of SQL data with Cypher query pushdown, enabling graph-native querying without physical data movement.

Hierarchical URIs enable efficient post-filtering in Lucene queries using starts_with predicates, avoiding expensive full-collection scans. Variable-length Cypher paths with parameterized depth (e.g., (lib)-[:has*0..2]->(node)) provide flexible tree traversal without hardcoding navigation logic. The Leiden community detection algorithm runs on in-memory graph projections, with the conductance metric quantifying cluster tightness through the ratio of internal to external edge density.

Document loading strategy depends critically on source structure. Deterministic loading uses regex and plain NLP on well-structured PDFs with explicit section markers, while messier sources may require LLM-based extraction or Grok patterns for entity recognition. The deterministic approach is preferred when applicable due to speed, cost, and reproducibility advantages. Semantic expansion in full-text search requires careful prompt engineering to generate domain-appropriate synonyms without excessive query broadening.

The Neo4j CLI tool and skill-based agent architecture enable dynamic Cypher query generation without relying on outdated Stack Overflow examples. The Neo4j Cipher skill and GDS skill provide current syntax and algorithm documentation, supporting model-agnostic agent development across different LLM providers. This structured skill definition approach is preferred over free-form text-to-Cypher due to better constraint specification and access to up-to-date algorithm knowledge.

5. Discussion

The research demonstrates that graph-based semantic architectures address fundamental limitations in current AI agent systems for multi-modal data retrieval. The three shapes - Connection, Table of Contents, and Themes - provide complementary capabilities: deterministic navigation through metadata and hierarchies, flexible keyword search with semantic expansion, and unsupervised pattern detection for global analysis. These capabilities enable both tactical queries (e.g., "For this VIN with this DTC code, what fix worked on similar vehicles?") and strategic queries (e.g., "Are there mismatches between documented procedures and field work orders?") that require proving negatives or identifying gaps.

The strategic positioning of graphs as semantic layers rather than data stores represents a pragmatic deployment model for enterprise environments. As articulated in the implementation philosophy, full ETL into graph databases is justified primarily when graph-specific algorithms - shortest path, clustering, graph embeddings - are required for the application domain: "If you have a supply chain for example and you need to find an optimal route from point A to B and you have to do these variable length, shortest path type of calculations. A graph database is very good for that." For most retrieval applications, metadata-only semantic layers provide sufficient navigation capability without the operational complexity of data migration.

Several areas warrant further investigation. Document versioning and authority tracking emerge as critical concerns: "The biggest problem is when you have documents that are outdated or drifted, right? So I've thought about, well, maybe we need to add a last updated date or something or whether or not a document should be authoritative." Integration with change data capture systems could enable temporal graph analysis of documentation evolution. Additionally, the interaction between deterministic graph navigation and probabilistic LLM reasoning requires deeper analysis to establish optimal division of labor between symbolic and neural components.

6. Conclusion

This research establishes that graph-based semantic architectures provide essential navigation capabilities for AI agents operating across multi-modal data environments. The Connection Shape enables reliable text-to-SQL generation through metadata graph navigation, the Table of Contents Shape provides deterministic hierarchical document traversal with cross-document linking, and the Themes Shape enables unsupervised pattern detection for estate-level analysis. These approaches collectively address the limitations of text-to-SQL and vector search at scale, particularly for queries requiring comprehensive coverage or negative evidence.

The practical implications are significant for enterprise AI deployments. Metadata-only semantic layers avoid data migration complexity while providing sufficient context for query generation. Deterministic document loading with hierarchical URIs enables reproducible navigation and efficient filtering. Community detection algorithms identify documentation gaps and organizational patterns without requiring LLM-based labeling. Organizations implementing AI agent systems for technical support, regulatory compliance, or knowledge management should consider graph-based semantic layers as foundational infrastructure, complementing rather than replacing traditional relational and vector search capabilities. Future work should focus on temporal graph analysis for document versioning, optimization of hybrid symbolic-neural reasoning strategies, and evaluation frameworks for measuring comprehensive retrieval quality beyond traditional precision-recall metrics.

Sources

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub