feat: durable persistence + verifiable audit log + operator CLI (#126, #127, #147, #124)

[dgenio]

What changed

A single coherent feature set — "make the audit trail durable, then provable, then operable" — implementing the four issues from the triage's recommended group. The four are mutually dependent (#127 builds on #126; the CLI reads #126/#127; doctor and audit share one CLI skeleton), so they belong in one PR.

#126 — Pluggable persistence for the stateful stores

• New weaver_kernel.stores package with TraceStoreProtocol, RevocationStoreProtocol, HandleStoreProtocol (mirroring the existing Driver/PolicyEngine protocol seams).
• Durable backends, stdlib-only (sqlite3/json, no new runtime dep): SQLiteTraceStore, JsonlTraceStore (append-only), SQLiteRevocationStore.
• In-memory TraceStore / HMACTokenProvider remain the defaults — opt in via constructor injection.
• Revocation extracted behind RevocationStoreProtocol on HMACTokenProvider; a token revoked through SQLiteRevocationStore stays revoked after a process restart.

#127 — Hash-chained, verifiable audit log

• Persisted traces carry a prev_hash/record_hash envelope (HMAC-SHA256 keyed by WEAVER_KERNEL_SECRET), separate from ActionTrace semantics; the wrapped payload is the redaction-safe export_action_trace shape, so chaining adds no field the trace didn't already hold (I-01 preserved).
• verify_chain() detects mutation, insertion, deletion, and reordering, reporting the first divergent record. SQLiteTraceStore.prune(before=...) enforces retention while keeping the retained suffix verifiable via a checkpoint.
• Honest scope documented in docs/security.md: tamper-evidence, not non-repudiation.

#147 + #124 — weaver-kernel operator CLI

• weaver-kernel audit list|show|verify|export over a persisted store: filters (--principal/--capability/--status/--since/--until/--limit), --json on every subcommand, redaction-safe output by construction, non-zero exit on tamper.
• weaver-kernel doctor: environment + optional-extras checks plus token sign/verify and audit-chain self-test vectors; exits non-zero only on a real error, warns (not fails) on a missing secret.
• argparse, stdlib-only, registered via [project.scripts]. Documented in new docs/cli.md.

Supporting changes

• Extracted HMAC secret loading to weaver_kernel._secrets (shared by token signing and audit hashing; avoids a tokens↔stores import cycle).
• Kernel(trace_store=...) typed against TraceStoreProtocol.
• Incidental hardening fix: the existing test_handle_expand_never_exceeds_grant property test surfaced a latent grant-cap bypass in HandleStore.expand — a negative limit (with max_rows=0) sliced rows[0:-1] and returned a row. Negative offset/limit are now rejected with INVALID_CONSTRAINT, plus a deterministic regression test. Unrelated to the persistence work but fixed here because it blocked CI and is a real grant-enforcement bug.
• Docs (architecture.md persistence section, security.md integrity model, new docs/cli.md, AGENTS.md map), CHANGELOG.md (0.11.0), and an offline examples/persistent_audit_demo.py wired into make example.

Why

These four issues form the triage's recommended combined group: a durable, verifiable, operable audit trail. They share the same modules (trace.py/new stores/, tokens.py revocation, a new CLI) and a hard dependency chain, so shipping them together avoids building-then-re-touching the same seams.

How verified

make ci (the single authoritative command) — green end-to-end:

• fmt-check (ruff format) — clean
• lint (ruff) — All checks passed!
• type (mypy --strict, 54 source files) — Success: no issues found
• test — 639 passed, 1 skipped (≈46 new tests across chain, SQLite, JSONL, CLI, doctor, handle regression)
• example — all examples run offline, including the new persistent_audit_demo.py

New tests prove the headline acceptance criteria: SQLite/JSONL round-trip + persistence across reopen; chain detects mutation/insertion/deletion/reorder/wrong-secret; prune-then-verify; revoked token stays revoked for a fresh provider; CLI list/show/verify(tamper→exit 1)/export; doctor self-test vectors.

Scope notes / caveats

• Mode B scope decision: all three store protocols are defined and exported, and SQLite/JSONL trace + SQLite revocation backends ship, but a persistent HandleStore backend is intentionally deferred to a follow-up. Handles are short-lived, TTL-bounded result caches and the most isolated from the audit/verify/CLI spine; the protocol is defined so it can be added later without a breaking change. Worth a follow-up issue.
• --status (not --decision allow|deny|ask) is the audit filter because denials are never traced (policy gates before invocation, per I-02) — code-true to the data model; documented in docs/cli.md.
• Integrity is tamper-evidence, not non-repudiation; the same secret signs tokens and the chain. Adjacent to the broader hardening issue security: production-hardening roadmap + ship the first piece (token signing/encryption, pluggable secrets/auth) #103.
• No back-compat shims were added (per the request); one existing token test was updated to follow the relocated secret-loading internals.

Closes #126, #127, #147, #124.

https://claude.ai/code/session_014vcsXSCqnFprxCkMiBQGtY

---

Generated by Claude Code

[Copilot]

Pull request overview

This PR delivers a cohesive “durable → verifiable → operable” audit trail feature set: pluggable persistence backends for traces/revocation, a hash-chained audit envelope with verification/pruning, and an operator-facing weaver-kernel CLI (audit + doctor). It also includes a handle-store constraint hardening fix (negative pagination) plus docs/examples/releases updates to ship the new functionality.

Changes:

• Add weaver_kernel.stores with protocol seams and stdlib-only durable backends (SQLite + JSONL), plus durable token revocation.
• Implement hash-chained trace records with verify_chain() and SQLite pruning checkpoints; expose chain verification via CLI.
• Add weaver-kernel CLI (audit list|show|verify|export, doctor) and update docs/examples/changelog/versioning.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/test_tokens.py	Adjusts dev-secret warning test to the new shared secret loader module.
tests/test_stores_sqlite.py	Adds SQLite store tests: round-trip, reopen persistence, chain verification, pruning, revocation durability.
tests/test_stores_jsonl.py	Adds JSONL trace store tests including tamper detection and blank-line handling.
tests/test_handles.py	Adds regression tests rejecting negative offset/limit in handle expansion.
tests/test_cli_doctor.py	Adds tests for weaver-kernel doctor text and JSON modes.
tests/test_cli_audit.py	Adds tests for weaver-kernel audit list/show/verify/export behaviors and tamper exit codes.
tests/test_audit_chain.py	Adds unit tests for record hashing, chain verification, and pruning checkpoint logic.
src/weaver_kernel/tokens.py	Refactors secret loading + extracts revocation behind a protocol with durable backend support.
src/weaver_kernel/stores/sqlite.py	Implements SQLiteTraceStore + SQLiteRevocationStore including chain and pruning checkpoint.
src/weaver_kernel/stores/memory.py	Adds InMemoryRevocationStore (default revocation backend for tokens).
src/weaver_kernel/stores/jsonl.py	Implements append-only JSONL trace store with chain support.
src/weaver_kernel/stores/audit_chain.py	Adds canonical hashing, TraceRecord, and verify_chain() implementation.
src/weaver_kernel/stores/_trace_codec.py	Implements encode/decode between ActionTrace and the persisted/export payload shape.
src/weaver_kernel/stores/_protocols.py	Defines TraceStoreProtocol, RevocationStoreProtocol, HandleStoreProtocol.
src/weaver_kernel/stores/init.py	Public export surface for store protocols, backends, and chain helpers.
src/weaver_kernel/kernel/_invoke.py	Retypes trace recording helpers to accept TraceStoreProtocol.
src/weaver_kernel/kernel/init.py	Allows injecting any TraceStoreProtocol into Kernel.
src/weaver_kernel/handles.py	Rejects negative pagination inputs to prevent grant-cap bypass.
src/weaver_kernel/cli/_doctor.py	Implements weaver-kernel doctor preflight checks and self-test vectors.
src/weaver_kernel/cli/_audit.py	Implements weaver-kernel audit command group for listing/showing/verifying/exporting traces.
src/weaver_kernel/cli/init.py	Adds CLI entry point, parsing, and kernel-error handling.
src/weaver_kernel/_secrets.py	Centralizes secret resolution with one-time dev fallback warning.
src/weaver_kernel/init.py	Re-exports new stores/protocols/verification helpers from the top-level package.
pyproject.toml	Bumps version to 0.11.0 and registers weaver-kernel console script.
Makefile	Adds the new persistent audit demo to make example.
examples/persistent_audit_demo.py	Adds an offline demo of durable audit + tamper detection + durable revocation.
docs/security.md	Documents the hash-chain integrity model and its limits (tamper-evidence, not non-repudiation).
docs/cli.md	Adds CLI documentation for audit and doctor.
docs/architecture.md	Documents protocol-based persistence seams and backend selection guidance.
CHANGELOG.md	Adds 0.11.0 release notes covering persistence, chain verification, CLI, and related changes.
AGENTS.md	Updates repo layout map and doc index to include stores/ and cli/.