Skip to content

Minu476/NexusVerifier

BranchesTags

Repository files navigation

NexusVerifier

NexusAgent is a graph-augmented Lean 4 verification pipeline for evaluating, filtering, and indexing formally verified proof artifacts from the Formal Conjectures benchmark corpus (arXiv:2605.13171).

Independence notice: This is an independent research project by Nasser Towfigh. It is not affiliated with, endorsed by, or derived from Google DeepMind or any of its products. It uses the Formal Conjectures benchmark (arXiv:2605.13171) released by Google DeepMind under Apache 2.0.

Architecture

Lean 4 Proof Parts
        │
        ▼
  NexusAgent Pipeline
        │
   ┌────┼────────────┐
   ▼    ▼            ▼
Axiom  Holdout     Neo4j
Check  Filter   Fossil Vault
                     │
                     ▼
           Verified Benchmark Results

Research status

This repository is an experimental research infrastructure project.

The FC100 benchmark corpus contains previously solved, non-open mathematical results with existing Lean formalizations. NexusAgent evaluates verification policy, axiom provenance, holdout isolation, and reusable proof indexing over this corpus.

This repository does not claim novel proofs of open Erdős problems.

Why this matters

Large language models can generate syntactically valid Lean proofs that rely on weakly trusted native compilation paths, or leak benchmark information through related declarations from the same problem file.

NexusAgent addresses this by:

  • Mechanically verifying proof artifacts using lake env lean + #print axioms
  • Filtering by axiom provenance — rejecting proofs that depend on native compilation (Lean.ofReduceBool, Lean.trustCompiler)
  • Enforcing declaration-family holdout isolation — when one declaration from a problem file is held out, all sibling declarations from the same parent are automatically excluded, preventing leakage between related theorems
  • Storing reusable verified subgoals in a graph-backed knowledge base (the "fossil vault") for future retrieval and tactic reuse

This is the infrastructure layer that separates rigorous formal verification from superficially plausible proof generation.

What this is

NexusAgent is theorem-proving infrastructure. It provides:

Capability Description
Benchmark reproducibility Deterministic, containerised verification of Lean proof artifacts
Axiom filtering Mechanical rejection of proofs depending on native compilation
Proof provenance Every verified part is tagged with its full axiom closure
Holdout isolation Declaration-family holdout prevents cross-theorem leakage
Graph reuse Verified sub-goals indexed in Neo4j for future proof retrieval

Benchmark results — FC100 (ingest-parts dry-run)

Evaluated on 10 sampled parts from the FC100SolvedSet1 corpus — 100 non-open problems with known sorry-free Lean 4 proofs — under a restricted axiom policy (native_decide → reject):

Status Count Meaning
Verified 7 Lean proof accepted; axiom closure ⊆ {propext, Classical.choice, Quot.sound}
Rejected 3 Proof depends on native compilation axioms (decide +native)
Excluded 0 (1 with holdout) Removed by declaration-family holdout policy

Holdout test: Excluding Erdos1074.erdos_1074.variants.EHSNumbers_init automatically excludes erdos_1074.variants.mem_pillaiPrimes (same parent erdos1074) — confirmed 6 passed, 3 rejected, 1 excluded.

The pipeline enforces declaration-family holdout isolation to prevent leakage between related theorems originating from the same benchmark problem file.

Provenance: The 7 verified parts are pre-existing Lean 4 proof terms from the FC corpus — proofs authored by the FC benchmark contributors, not generated by NexusAgent. NexusAgent's role is mechanical verification (axiom-closure filtering + holdout isolation), not proof synthesis.

FC100SolvedSet1 is a corpus of already-solved, non-open problems with known Lean formalizations; this pipeline verifies them mechanically and filters by axiom strength.

Stack

Layer Technology
Formal language Lean 4 (v4.27.0) + Mathlib
Proof verification lake env lean + #print axioms
Graph backend Neo4j 5 (Community via Docker; Enterprise locally)
Agent pipeline C# / .NET 10
Tests 96 unit tests (xUnit)
Container Docker (multi-stage, ~1.6 GB)

Reproducibility

Verified with:

  • Lean v4.27.0 (elan-managed, baked into Docker image)
  • Mathlib pinned via lake-manifest.json in formal-conjectures/
  • Neo4j 5.x
  • .NET 10
  • Docker Desktop 4.x (macOS arm64)

Quickstart (Docker)

Prerequisites

# 1. Clone and build the formal-conjectures project (downloads ~1 GB Mathlib)
git clone https://github.com/google-deepmind/formal-conjectures
cd formal-conjectures
lake update && lake build        # ~20 min on first run
cd ..

# 2. Configure environment
cp .env.example .env
# Edit .env — set NEO4J_PASSWORD and optionally LLM API keys

# 3. Build the NexusAgent image
docker build -t nexus-agent:latest .

Run a dry-run verification

docker run --rm \
  -v /path/to/formal-conjectures:/formal-conjectures \
  -v "$(pwd)":/workspace:ro \
  -e NEXUS_NEO4J_URI=bolt://host.docker.internal:7687 \
  -e NEXUS_NEO4J_PASSWORD=your_password \
  -e NEXUS_NEO4J_DATABASE=nexusdb-snapshot \
  -e NEXUS_LEAN_PROJECT=/formal-conjectures \
  -e NEXUS_PARTS_NATIVE_DECIDE=reject \
  nexus-agent:latest \
  ingest-parts --from-json /workspace/parts.json --dry-run

Expected output:

[ingest-parts] 10 parts from parts.json  sinks=[fossil]  (DRY RUN — no writes)
  FAIL  erdos_1148.variants.lower_bound  — native axioms (policy=reject): [...]
  PASS  [Weaker] erdos_647.variants.twenty_four  axioms=[propext, ...]
  ...
[ingest-parts] Done: 7 passed, 3 rejected, 0 excluded (holdout).

With holdout exclusion

echo "Erdos1074.erdos_1074.variants.EHSNumbers_init" > /tmp/fc100_targets.txt

docker run --rm \
  -v /path/to/formal-conjectures:/formal-conjectures \
  -v "$(pwd)":/workspace:ro \
  -v /tmp/fc100_targets.txt:/tmp/fc100_targets.txt:ro \
  -e NEXUS_NEO4J_URI=bolt://host.docker.internal:7687 \
  -e NEXUS_NEO4J_PASSWORD=your_password \
  -e NEXUS_NEO4J_DATABASE=nexusdb-snapshot \
  -e NEXUS_LEAN_PROJECT=/formal-conjectures \
  -e NEXUS_PARTS_NATIVE_DECIDE=reject \
  nexus-agent:latest \
  ingest-parts --from-json /workspace/parts.json \
               --exclude-targets /tmp/fc100_targets.txt \
               --dry-run

Expected: Done: 6 passed, 3 rejected, 1 excluded (holdout).

Benchmark results — Proof search

NexusAgent also runs as an active proof-search agent: given a theorem statement from the FC corpus, it attempts to produce a sorry-free Lean 4 proof using its graph-guided planner and LLM tiers. Results below are from May 2026 benchmark runs; full telemetry (per-problem outcomes, cost, turn counts, fossil hits) is in data/results/ and summarised in SOLVED_PROBLEMS.md.

Note on naming: The directory names erdos_phase8/ and erdos_phase9_ams5/ continue the build-phase numbering from SPEC.md §5 — Phases 0–7 were infrastructure construction (LeanOracle, Neo4j schema, HallucinationGate, ProofCartographer, etc.). Phase 8 was the first live benchmark execution; Phase 9 was the expanded AMS-5 corpus run.

Terminology: Compiled = Lean accepted the file (may include sorry). Sorry-free = axiom closure contains no sorryAx; the proof is mechanically complete. All sorry-free proofs were verified by #print axioms inside the pipeline.

Benchmark run 1 — Pilot (erdos_phase8/, 10 problems)

Metric Value
Problems attempted 10
Compiled (Lean accepts file) 10 / 10
Sorry-free proofs 7 / 10
Sorry-bearing (compiled, incomplete) 3 / 10
Total LLM cost $0.44

Sorry-free: #109 (Erdős sumset conjecture), #139 (Szemerédi's theorem), #194, #219 (Green–Tao), #228 (flat Littlewood polynomials), #239, #250.

Benchmark run 2 — AMS-5 corpus (erdos_phase9_ams5/, 66 problems)

Metric Value
Problems attempted 66
Compiled (Lean accepts file) 53 / 66 (80%)
Sorry-free proofs 43 / 66 (65%)
Aborted 12
Budget exhausted 1
Total LLM cost $4.81

See SOLVED_PROBLEMS.md for per-problem mathematical descriptions of the 15 highlight results, and data/results/ for HTML and JSON reports with full per-problem telemetry.

Verification policy

The axiom policy is controlled by NEXUS_PARTS_NATIVE_DECIDE:

Value Behaviour
reject (default) Parts using native_decide / Lean.trustCompiler are rejected outright
warn Parts are accepted but flagged in output
allow No filtering applied

A proof is Verified when its axiom closure is a subset of {propext, Classical.choice, Quot.sound} — the standard Lean 4 logical foundation. Any additional axioms (especially native compilation bridges) indicate reliance on unverified reduction paths and are rejected under the reject policy.

Scope tags ([Weaker], [Instance], [Part]) are independent of verification status and indicate how the proven statement relates to the original conjecture — a full-strength theorem can be axiom-clean, and a weaker variant can still be axiom-unsound. These are reported alongside the verification result, not as a sub-grade of it.

Environment variables

Variable Default Description
NEXUS_NEO4J_URI bolt://localhost:7687 Neo4j Bolt URI
NEXUS_NEO4J_USER neo4j Neo4j user
NEXUS_NEO4J_PASSWORD (empty) Neo4j password
NEXUS_NEO4J_DATABASE neo4j Database name
NEXUS_LEAN_PROJECT (required) Path to built formal-conjectures checkout
NEXUS_PARTS_NATIVE_DECIDE reject reject (default) fails proofs using decide +native; warn accepts but flags
GOOGLE_API_KEY (optional) Gemini hallucination gate
DASHSCOPE_API_KEY (optional) Qwen cloud gate

Building natively (.NET 10)

cd NexusAgent
dotnet build NexusAgent.sln

# Run all unit tests (96 tests, no external deps)
dotnet test NexusAgent.Tests/NexusAgent.Tests.csproj \
  --filter "Category!=Integration" -v q

Project layout

NexusAgent/
  NexusAgent.Core/            # Configuration, Neo4j client, LeanOracle, planning
  NexusAgent.Cli/             # CLI entry point (ingest-parts, bench, …)
  NexusAgent.VerifiedParts/   # AxiomChecker, VerifiedPartIngestor, holdout logic
  NexusAgent.MathlibIngestor/ # Mathlib tactic graph ingestor
  NexusAgent.Tests/           # 96 unit tests (xUnit)
formal-conjectures/           # Separate clone of google-deepmind/formal-conjectures (not a submodule)
data/                         # Benchmark input files and results
docs/                         # Architecture notes, Neo4j schema, Cypher queries

Future work

  • Graph-guided proof retrieval — query the fossil vault for reusable sub-goals by goal-state similarity
  • Tactic reuse — surface verified tactic sequences from the knowledge graph during proof search
  • Proof clustering — group structurally similar proofs to detect common patterns
  • Fossil-vault retrieval policies — configurable strategies for querying verified sub-goals
  • Automated theorem search — use the hypergraph structure to identify candidate lemmas for new conjectures
  • Expanded benchmark coverage — evaluate over the full FC100 corpus and beyond

License

Apache 2.0. See LICENSE.

The formal-conjectures/ directory is governed by its own license (Apache 2.0).

About

Graph-augmented formal proof verification pipeline for the DeepMind Formal Conjectures benchmark

Resources

Readme

License

View license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

NexusVerifier v0.1.0 — Initial Public Release Latest
Jun 1, 2026

Packages

 
 
 

Contributors

Languages