From policy text to provable action across every dependent data and AI asset.
PolicyLineage is an evidence-first regulatory compiler for DataHub. It turns a new supported policy pack into atomic, testable rules; intersects those rules with live metadata and multi-hop lineage; separates exposure from evidence confidence; and prepares human-approved governance actions that write the decision back to DataHub.
It was built from scratch and submitted to Build with DataHub: The Agent Hackathon 2026.
A policy change rarely names the internal dataset, model, pipeline, dashboard, research project, or owner that must act. Conventional compliance tools track obligations in one system while data catalogs track technical dependencies in another. The costly gap is the translation between them.
PolicyLineage closes that gap:
Primary policy sources
↓
Atomic rules + exceptions + counter-evidence
↓
DataHub schemas + governance signals + multi-hop lineage
↓
ALLOW / REVIEW / BLOCK + affected owners + remediation
↓
Human approval → governed DataHub writeback → reusable evidence bundle
DataHub already provides excellent catalog search and lineage. PolicyLineage uses those capabilities as the organization graph and adds a missing policy decision layer:
- Three graphs, one verdict: policy obligations, organizational lineage, and decision evidence.
- Falsifiable by design: supporting evidence and exception/counter-evidence remain visible.
- Deterministic gate: an LLM may extract proposed rules; a testable rule engine owns the verdict.
- Two heuristic indexes, never conflated: operational impact priority and evidentiary coverage are separate, inspectable aids—not probabilities or legal conclusions.
- Closed-loop governance: proposed tags, structured properties, and a decision document return to DataHub after explicit human approval.
- Time-aware: policy, asset, and decision versions produce a digestible evidence bundle that can later be invalidated when upstream facts change.
The repository already contains a zero-secret synthetic scenario, Operation Kintsugi. It models controlled quantum research data flowing through a feature table and ML model into an externally accessible collaboration job. A synthetic notice includes both a restriction and a possible fundamental-research exception.
The hackathon release deliberately ships one audited deterministic policy pack. It is not presented as general-purpose legal-language extraction; additional expert-labeled packs are part of the post-hackathon product path.
Run it:
uv sync --all-extras
uv run policylineageOpen http://127.0.0.1:8000, press Compile against DataHub, and inspect the decision, lineage exposure, both sides of the evidence ledger, and approval-gated writeback plan.
The public static rehearsal runs the same compiler-generated BLOCK and ALLOW evidence bundles without credentials. Use the FastAPI app for durable approval and live DataHub writeback.
Judges can use the one-page Judge Guide to verify every official criterion against a test, captured output, public workflow, or source file. The matching machine-readable evidence index keeps the proof map independently inspectable.
The decision exposes the exact multi-hop impact graph, counter-evidence, approval-bound writeback plan, and superseded digest rather than collapsing them into a chatbot answer. See the full screenshot sequence.
The FastAPI app also exposes a durable Decision Registry at GET /api/decisions and a
complete audit record at GET /api/decisions/{decision_id}. Each record keeps the immutable
bundle, original analysis context, attributable approval/writeback state, and supersession
link together. These read-only endpoints use the same bearer guard as live DataHub access.
PolicyLineage now includes an authenticated, PostgreSQL-ready Control Plane for the SaaS path:
- isolated workspaces with hashed, revocable API keys and four least-privilege roles;
- searchable decision records, immutable evidence bundles, and attributable audit events;
- a durable revalidation queue that detects changed policy/catalog evidence and links superseding decisions; and
- the existing preview, human-approval, digest, and idempotency gates around every catalog mutation.
The anonymous synthetic hackathon rehearsal remains the default. Enable the tenant-aware service with
POLICYLINEAGE_CONTROL_PLANE_ENABLED=true and a bootstrap secret. See the
Control Plane deployment and API guide, including the separate PostgreSQL
Render Blueprint.
Watch the 2:44 audited demo on YouTube or follow the exact recording script.
Or deploy the deterministic public rehearsal with the included Render Blueprint:
Run checks:
uv run ruff check .
uv run pytest
uv run policylineage-evaluate --output examples/evaluation-report.jsonThe included 14-case expert-designed synthetic regression set currently records 100% verdict accuracy, affected-asset precision and recall, exception handling, replay stability, incomplete-graph safe routing, approval gating, and identity-attribute invariance. These measure the deterministic compiler boundary; they are not claims of legal accuracy.
The deterministic fallback makes the judge experience reliable. A real DataHub 1.6.x seed, official Streamable HTTP MCP client, explicit partial-graph handling, digest-bound human approval, typed metadata writeback, and durable idempotency ledger are also implemented. Follow the live DataHub setup.
Live DataHub mode additionally requires an API bearer token and a server-derived approver identity. The public synthetic rehearsal remains credential-free and cannot mutate a catalog.
The manual Live DataHub integration GitHub workflow starts DataHub Core in Docker,
seeds the scenario, launches the official MCP server, performs approved writeback, and
uploads its evidence report so the full integration can be independently replayed.
The latest clean-room run is visible on the public
Live DataHub workflow;
the checked-in evidence snapshot records the exact source run. It resolved
8 assets and 7 lineage edges, produced a 6-asset BLOCK, executed all three
approved mutation tools, and verified the resulting tag and typed properties. A compact
recorded evidence summary is committed for judges.
Sample outputs are available in examples/, including the stable Operation
Kintsugi decision summary and the exact MCP call plan.
The conditional hackathon probability model is reproducible and explicitly non-predictive:
uv run policylineage-award-model --output examples/award-probability.jsonIts assumptions, decision use, SaaS wedge, pricing hypotheses, KPI contract, and 90-day path are documented in the award and SaaS strategy.
The requirement-by-requirement submission checklist links every completed claim to public evidence and keeps account-only actions explicit.
src/policylineage/
├── api.py # FastAPI demo surface
├── demo.py # synthetic policy + DataHub-shaped scenario
├── domain.py # evidence, policy, catalog, and decision contracts
├── engine.py # deterministic rule + required-path evaluation and traversal
├── evaluation.py # reproducible expert-designed gold-set benchmark
├── policy_parser.py # auditable policy-text to typed-rule boundary
├── store.py # workspace, decision, audit, queue, and idempotency persistence
├── integrations/ # DataHub seed, MCP client, context collector, writeback gate
└── static/ # no-build interactive demo UI
tests/ # executable behavioral claims
docs/ # product, architecture, evaluation, and submission strategy
datahub/ # typed property definitions for the demo instance
examples/ # stable outputs judges can inspect without running the app
oss/ # validated auditable-data-change skill contribution
The upstream-ready Agent Skill lives at
oss/datahub-skills/skills/auditable-data-change.
It is also open for review as
datahub-project/datahub-skills#45.
PolicyLineage is decision support, not legal advice. It cites the source and locator for every extracted claim, exposes exceptions and unresolved evidence, requires responsible human approval for material actions, and never turns country or identity into a proxy for risk. Decisions apply to explicit policy conditions and observed dependencies.
Apache License 2.0. See LICENSE.