🕸️ Graph Engineering — The 5 Layers (Production Edition)

Premium Guide: https://shop.beacons.ai/aiengineeringinsider/6e19d580-1f8c-4bb7-a0f0-f1583141a0b8

preview: https://drive.google.com/file/d/1olbRpR6UwPIGuUWyPk4FXSPbrUuf3WiU/view?usp=sharing

"Architecture beats model choice."
GPT-3.5 in an agentic loop scored 95.1% on HumanEval vs GPT-4 zero-shot at 67.0% (Andrew Ng, 2026).

A production-grade Python & Streamlit reference implementation of the 5-Layer Graph Engineering Taxonomy, implementing the complete technical outline from graph-engineering-layered-outline.md.

---

📑 The 5-Layer Taxonomy

Every AI system can be analyzed through five nesting layers. You do not "graduate" from one layer to the next — you add a layer only when the layer below it exhibits a named, observable failure mode.

#	Layer	What it engineers	Core Question	Failure Mode That Forces Next Layer
1	Prompt	The single request	Am I asking well?	Model follows prompt, but lacks facts it couldn't know $\rightarrow$ Context
2	Context	What the model sees	Does it have the right info?	Model knows facts, but cannot execute code or save state $\rightarrow$ Harness
3	Harness	Tools, memory, scaffolding	Can it act on the world?	Single tool call produces unchecked errors $\rightarrow$ Loop
4	Loop	Repeat cycle one agent runs	When does it check work & stop?	Single loop context window overflows or lacks specialization $\rightarrow$ Graph
5	Graph	Multi-agent state & memory	Who does what, in what order, sharing what state?	Production ceiling reached; durable relational state required.

---

🏛️ System Architecture

graph-engineering-project/
├── .env                              # API Key configuration (GOOGLE_API_KEY, LLM_PROVIDER)
├── .streamlit/                       # UI styling & configuration
│   └── config.toml                   # Custom modern dark theme (#09090B zinc / violet gradient)
├── app.py                            # Streamlit Master Web Application (Parts I - V)
├── config/                           # System configuration
│   └── settings.py                   # Environment loader
├── requirements.txt                  # Production dependencies
├── run_e2e_tests.py                  # Background server & Playwright E2E runner launcher
├── src/                              # Core System Engine & Microservices
│   ├── llm_factory.py                # Model Provider Controller (Gemini 2.5 Flash default)
│   ├── mcp_service.py                # Model Context Protocol (MCP) JSON-RPC 2.0 Server
│   ├── graph_engine.py               # LangGraph state machine & NetworkX GraphRAG engine
│   └── chapters/                     # 16 Production Chapter Modules
│       ├── ch01_one_building_five_floors.py
│       ├── ch02_layer1_prompt.py
│       ├── ch03_layer2_context.py
│       ├── ch04_layer3_harness.py
│       ├── ch05_layer4_loop.py
│       ├── ch06_layer5_graph_overview.py
│       ├── ch07_diagnosing_layers.py
│       ├── ch08_staged_build_path.py
│       ├── ch09_graph_grounded_multiagent.py
│       ├── ch10_minimal_graph_schema.py
│       ├── ch11_extraction_agents.py
│       ├── ch12_graphrag_retrieval.py
│       ├── ch13_temporal_entity_resolution.py
│       ├── ch14_when_graph_earns_itself.py
│       ├── ch15_capstone_code_reviewer.py
│       └── ch16_production_readiness.py
└── tests/                            # Comprehensive Automated Test Suites
    ├── test_graph_engine_unit.py    # Pytest unit tests for llm_factory, MCP, and chapter imports
    ├── test_e2e_playwright.py       # Non-headless Playwright browser test suite
    └── screenshots/                  # Verified full-page PNG execution screenshots (16 chapters)

---

📚 Book Structure & Chapter Index

Part I — Why Layers

• Chapter 1 — One Building, Five Floors: The 5-layer map, the nesting property, and why architecture beats model selection.

Part II — The Five Layers

• Chapter 2 — Layer 1: Prompt: System prompts, zero/few-shot examples, formatting constraints, and prompt diagnosis.
• Chapter 3 — Layer 2: Context: Budgeting context windows, avoiding lost-in-the-middle degradation, and context boundaries.
• Chapter 4 — Layer 3: Harness: Tool execution schemas, deterministic python code anchors, and external system permission boundaries.
• Chapter 5 — Layer 4: Loop: Actor-critic reflection cycles, counter-metrics, stopping conditions, and Goodhart's law gaming prevention.
• Chapter 6 — Layer 5: Graph Overview: Evolution from loop $\rightarrow$ chain $\rightarrow$ network $\rightarrow$ graph; multi-agent coordination.

Part III — Using the Layers

• Chapter 7 — Diagnosing Which Layer You're On: Diagnostic flowcharts to locate failures at the exact layer before refactoring.
• Chapter 8 — Add-a-Layer Rule & Staged Build Path: Staged progression (Zero-shot $\rightarrow$ Reflection $\rightarrow$ Tool Use $\rightarrow$ Multi-Agent $\rightarrow$ Graph).

Part IV — Deep Dive: Graph Engineering (Layer 5)

• Chapter 9 — Graph-Grounded Multi-Agent Architecture: LangGraph state machine with typed state channels, conditional routing, and fan-out.
• Chapter 10 — The Minimal Graph Schema: 5 Node types (Entity, Claim, Source, Artifact, Run) and 5 Edge types (mentions, supports, contradicts, derived_from, supersedes). Demonstrates the additive-write principle.
• Chapter 11 — Building the Graph From Raw Data: Schema-validation gates, provenance tracking (derived_from), and multi-triple extraction agents.
• Chapter 12 — GraphRAG: Retrieval Over the Graph: Question routing (Vector for single-fact vs Graph for multi-hop causal chains), entry point identification, and traversal synthesis.
• Chapter 13 — Temporal Memory & Entity Resolution: Bi-temporal validity (valid_from / valid_until), canonical ID resolution, and inspectable resolution merge logs.
• Chapter 14 — When the Graph Earns Itself: Evaluating the 3 criteria (Session Persistence, Cross-Agent Coordination, Traceability) using LLM structured output.
• Chapter 15 — Capstone: Graph-Grounded Code Reviewer: Staged benchmark climb (55% $\rightarrow$ 95%), repeat pattern tracking across PRs, and deterministic test execution anchors.

Part V — Production

• Chapter 16 — Production Readiness, by Layer: Per-layer production audits with Pydantic structured output, identifying security, latency, and reliability gaps.

---

⚡ Quick Start & Installation

1. Requirements & Prerequisites

• Python 3.9+
• Google Gemini API Key (GOOGLE_API_KEY)

2. Environment Setup

Create a .env file in graph-engineering-project/:

GOOGLE_API_KEY=your_gemini_api_key_here
LLM_PROVIDER=gemini

3. Run the Streamlit Master Application

Launch the interactive web application:

cd graph-engineering-project
../.venv/bin/streamlit run app.py

Open your browser at http://localhost:8501.

---

🧪 Testing & Verification

Unit Tests

Verify module imports, Model Provider Controller, and MCP JSON-RPC service:

cd graph-engineering-project
../.venv/bin/python3 -m pytest tests/test_graph_engine_unit.py -v

Playwright End-to-End (E2E) Browser Suite

Runs a non-headless Chromium browser session that navigates all 16 chapters, clicks action buttons, waits for real Gemini 2.5 Flash LLM responses, verifies DOM output, and captures full-page screenshots:

cd graph-engineering-project
../.venv/bin/python3 run_e2e_tests.py

All 16 full-page PNG screenshots will be stored in tests/screenshots/.

---

⚙️ Core Engineering Principles Implemented

1. No Hardcoded/Fabricated Responses: Every single chapter invokes get_llm() from src/llm_factory.py with real prompts and structured outputs.
2. Pydantic Validation: All structured outputs use pydantic.BaseModel and .with_structured_output().
3. Additive-Write Principle: Knowledge graph claims are never overwritten silently; new claims are linked via supersedes edges.
4. Deterministic Anchors: LLM reasoning is anchored by exogenous Python code execution before final approval.
5. Model Context Protocol (MCP): Implements JSON-RPC 2.0 resources and reflection tools adhering to MCP standards.
6. Graceful Error Handling: Runtime exceptions are caught and surfaced via st.error() without swallowing stack trace context.

---

Part of the AI Engineering Insider Series (2026 Edition).

graph-engineering-project