Skip to content

grammy-jiang/intelligence-analysis-agent

Repository files navigation

intelligence-analysis-agent

CI codecov Python License: MIT Ruff pre-commit FastMCP

A composed AI agent that runs the intelligence-analysis / analytic-tradecraft workflow — frame a question, enumerate competing hypotheses, weigh evidence in an ACH matrix ranked by disconfirmation, get independent bias / method / calibration critique, and commit a calibrated judgment only after a human approves it. Thesis: augment, not replace — the agent structures and challenges the reasoning; the human owns the judgment.

Grounded in a distilled corpus of canonical works (Heuer, Kahneman, Tetlock ×2, Jervis, Kent, CIA SAT Primer, FM 2-22.3, Masterman). Built by, and consuming reviewers from, the subagent-factory.

Status — Phase 3 complete (MCP state layer); Phase 4 built, live collection gated

The prose core proved out first (MVP-0 → Phase 2 gates passed), then the persistent state/compute layer was built on top. The whole thing still runs a case as prose in-context; the MCP layer adds a durable, tamper-evident store and compute for cross-case learning.

Layer What State
Skill (the method) structured-analysis — the 11-step workflow the main agent runs in-flight ✅ built
"Doing" skills source-evaluation (Step 3) + calibrated-forecasting (Step 9) — the doing counterparts to the reviewer critics ✅ built (Phase 2)
Subagents (the critics) bias-perception-reviewer, analytic-method-reviewer, calibration-forecasting-reviewer, deception-detection-reviewer — read-only, delegated critique ✅ deployed
Human gate + loop-back no judgment is final until a human approves; failed review loops back ✅ in the skill
MCP (state + compute) ach-engine, calibration-tracker, evidence-ledger — durable, hash-chained, tamper-evident ✅ built (Phase 3), 149 tests
OSINT + deception osint-toolkit (sole egress, SSRF-guarded) + osint-investigation skill + deception-detection-reviewer 🟡 built (Phase 4); live collection gated off (OSINT_LIVE=0)

A case still runs in-context as prose; the MCP state layer is opt-in durability, not a rewrite of the method. External collection is off by default and confined to a single audited egress surface.

Architecture

How a case flows at runtime — the agent structures and challenges the reasoning, the reviewer subagents critique it, the MCP layer persists it, and no judgment commits until a human approves:

flowchart TB
    Analyst["Human analyst — owns the judgment"]
    subgraph Agent["Claude Code · structured-analysis skill"]
        Skill["11-step workflow:<br/>frame → hypotheses → ACH → judgment"]
        Doing["source-evaluation · calibrated-forecasting"]
    end
    subgraph Critics["Reviewer subagents · read-only critique"]
        Bias["bias-perception"]
        Method["analytic-method"]
        Calib["calibration-forecasting"]
        Decep["deception-detection"]
    end
    subgraph MCP["MCP state layer · append-only · hash-chained"]
        ACH["ach-engine"]
        Ledger["evidence-ledger"]
        Cal["calibration-tracker"]
    end
    OSINT["osint-toolkit<br/>sole egress · SSRF-guarded"]
    Web(["Public OSINT sources"])
    Gate{"Human gate:<br/>approve?"}
    Commit["Calibrated judgment committed"]

    Analyst --> Skill
    Skill --> Doing
    Skill --> Critics
    Skill <--> MCP
    Web <-->|OSINT_LIVE gated| OSINT
    OSINT -->|ungraded proposals| Ledger
    Skill --> Gate
    Critics --> Gate
    Gate -->|approved| Commit
    Gate -->|failed review| Skill
Loading

Validation

  • MVP-0 / Phase 1 gate — PASS (docs/validation/results-2026-07-10.md, Heuer Iraqi-retaliation ACH): agent-assisted surfaced 5 hypotheses vs the baseline's 2; the ACH matrix demoted the lead hypothesis on a "dog that didn't bark" disconfirming item; and 3/3 deliberately injected flaws were each caught by the correct reviewer (vividness → bias, unstated assumption → method, overconfidence → calibration). The reviewers also caught a real, unplanted bookkeeping error in the hand-built matrix.
  • Phase 2 Brier gate — PASS (docs/validation/brier-forecasting-demo.md): assisted 0.091 vs a naive/overconfident baseline 0.247 (63% lower) on 8 resolved questions. Honest caveat recorded — this validates the mechanics + direction, not a blind benchmark (that needs a held-out feed, now possible via calibration-tracker).
  • Phase 3/4 tests — 149 passing (python3 -m pytest): server logic + cross-server wiring + OSINT egress and EXIF hardening.

Each design was gated through a factory advisor panel to must-fix = 0 before any code: Phase 3 over three rounds (mcp-protocol / mcp-security / mcp-quality / ai-agent-engineering); Phase 4 through a mandatory, blocking security review (mcp-security + application-security).

MCP servers (state + compute)

Four FastMCP servers, wired in .mcp.json. Every server keeps an append-only, hash-chained manifest and exposes a read-only verify_chain integrity tool (run at startup + anchored to an external head hash), so the store is tamper-evident.

Server Tools Grounding
calibration-tracker log_forecast, resolve_forecast, void_forecast, get_forecast, list_forecasts, get_calibration_report, verify_chain Tetlock — question + probability locked at commit, only the outcome is appended; Brier/calibration computed by the tool
evidence-ledger add_evidence, grade_evidence, update_grade, get_evidence, list_evidence, get_source_history, verify_chain, verify_signals_chain FM 2-22.3 — reliability/credibility on two independent axes; PII a required field; unredaction gated
ach-engine create_matrix, add_hypothesis, rate_cell, score_matrix, get_matrix, list_matrices, verify_chain Heuer ACH — ranks by least-inconsistency; score_matrix refuses any cell whose evidence lacks an analyst_confirmed grade (collect-then-grade hole closed)
osint-toolkit search, fetch, compute_hash, extract_exif, reverse_image_search, get_map_tile, propose_to_ledger, verify_chain The sole external-egress surface; per-hop SSRF guard (IANA blocklist + canonicalization + connection pinning), egress audit log, opaque artifact_ref, extract_exif RCE-hardened

Security posture (Phase 4 gate = unclassified / OSINT-only): public open-source data only; egress limited to allowlisted OSINT sources; no classified data at rest. judgment_source / analyst_id come from a trusted local binding, not an LLM argument — and are honestly bounded (skill-asserted, not server-verified). OSINT_LIVE=0 keeps all live connectors off until explicitly enabled.

Use it

Open Claude Code in this repo (the .claude/ here provides the skill + the reviewer subagents; .mcp.json provides the state servers), then:

/structured-analysis

or just describe an analytic question under uncertainty. The skill maintains a case workspace (question · hypotheses · evidence+grades · ACH matrix · assumptions · ranking · findings · judgment · assessment · indicators), delegates critique to the reviewer subagents on the raw case state, and stops for your approval before finalizing. Persisting a case to the MCP servers is optional.

The product is repo-local: the skill and the critic subagents are co-located in this repo's .claude/, so they must run together here (the orchestrator delegates to the reviewer subagents, which are not global). Work a case per session in this repo.

Layout

.claude/
  agents/     the 4 reviewer subagents (deployed from the factory — do not edit; re-export to update)
  skills/     structured-analysis (orchestrator) + source-evaluation + calibrated-forecasting
              + osint-investigation + the reviewers' own skill modules
mcp_servers/  ach_engine · calibration_tracker · evidence_ledger · osint_toolkit + common/staleness
docs/
  design/     phase3-mcp-design.md · phase4-osint-design.md (each gated to must-fix=0)
  validation/ the MVP-0/Phase-2 gates + run results + Brier tooling
tests/        234 tests — server logic + cross-server wiring + OSINT egress/EXIF + coverage
scripts/      check_mermaid.py (mermaid-cli compile) · check_egress_isolation.py · check_coverage.py

Development

Dependencies and the virtual environment are managed with uv. The environment is locked (uv.lock) and its Python is pinned (.python-version); create or refresh it with:

uv sync           # build .venv from uv.lock (runtime deps + the `dev` group)
uv run pytest     # run the suite in the locked env

The state MCP servers in .mcp.json launch via uv run, so uv sync must have been run first.

Linting and formatting run through pre-commit. The hooks cover every file type in the repo — Python (ruff lint + format), Markdown (markdownlint + a Mermaid compile check), and TOML / JSON / YAML plus whitespace hygiene:

uv run pre-commit install           # run the hooks on every git commit
uv run pre-commit run --all-files   # check the whole repo once

Diagrams in the docs are Mermaid fenced code blocks, rendered inline by GitHub. scripts/check_mermaid.py compiles every fenced mermaid block with the real Mermaid compiler (mmdc) — wired as a repo-local pre-commit hook and, because it needs Node + a headless browser, as its own CI job — so a diagram that doesn't render can't land in the docs. Ruff rules live in pyproject.toml; Markdown rules in .markdownlint-cli2.jsonc (the vendored .claude/ product is excluded from the Markdown linter, not from the Mermaid check).

CI (.github/workflows/ci.yml) runs on every push to master and every pull request: the pre-commit hooks above; the test suite on Python 3.11–3.13 (plus 3.14 as an allowed-to-fail preview) under coverage with a per-module gate (scripts/check_coverage.py) — core modules (the store engines, common, staleness, the egress SSRF guard, models) must clear 95% and every other module 90%, enforced per file so a well-covered module can't mask a thin one (currently every module is 96–100%); and an egress-isolation gate (scripts/check_egress_isolation.py) that fails the build if a data server (calibration-tracker / evidence-ledger / ach-engine) gains a direct network import — enforcing the design's "osint-toolkit is the sole egress surface" invariant.

Design of record

The what/why/decisions live in the factory repo under docs/intelligence-analysis/: BLUEPRINT-* (decisions + open questions), PIPELINE-grounded.md (the workflow, every step cites a source claim), DESIGN-SPEC-* (the how). The reviewer subagents are regenerated there and deployed here via cli export-deployable.

About

Intelligence-analysis agent — MCP state + OSINT layer (calibration-tracker, evidence-ledger, ach-engine, osint-toolkit)

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages