feat: machine-written evidence — Evidence Logger hook, citation corroboration, opt-in acceptance replay

SKILL.md

@@ -130,12 +130,13 @@ Selecting a profile: the user's words above, env `FABLE_ROUTING=quality|balanced
 
 ## Enforcement layer (hooks — mechanics in `hooks/README.md`)
 
-Four hooks turn the most-shirked rules into hard blocks. Armed **per project** by a `.fable/` directory (searched upward, bounded at the git root); without it they pass through silently. Pressure applies **per round** via `.fable/LEDGER.md`:
+Five hooks turn the most-shirked rules into hard blocks. Armed **per project** by a `.fable/` directory (searched upward, bounded at the git root); without it they pass through silently. Pressure applies **per round** via `.fable/LEDGER.md`:
 
 ```
 - [ ] 1. card (machine-checkable acceptance)   <- open: guards enforce
-- [x] 2. done -- evidence: pytest 21/21        <- [x] REQUIRES a substantive evidence note
+- [x] 2. done -- evidence: `pytest -q` 21/21   <- [x] REQUIRES substantive evidence; a cited `command` is checked against the machine-written evidence log
 - [~] 3. not this round -- deferred: reason
+REPLAY: on                                     <- optional: re-run cited acceptances before the round may end
 PAUSED: reason                                 <- a line anywhere: enforcement off
 ```
 
@@ -145,7 +146,8 @@ missing.)
 
 - **Spawn Guard** (PreToolUse Agent/Task/Workflow): blocks a detailed spawn while the ledger has no **open** cards — no ledger, and equally a ledger holding only a finished round's closed cards (design gate: new fan-out needs a live card) — and blocks any spawn requesting a **model stronger than the session's** (model ceiling — checked on the `model` param and `model:` literals in Workflow scripts; stays active even when paused, it protects quota, not workflow).
 - **Fail-Streak Reminder** (PostToolUse Bash, advisory): every 3rd consecutive failing command injects the attribution ladder — stops grinding on the wrong layer mechanically, not by willpower.
-- **Close Guard** (Stop): blocks ending the turn while open `- [ ]` items remain, **and** while any `- [x]` lacks an `-- evidence:` note (evidence-on-close: adjectives don't close cards).
+- **Evidence Logger** (PostToolUse Bash, passive): appends every command's real outcome (command, exit code, output tail) to `.fable/evidence.jsonl` — the machine-written record the Close Guard checks citations against. Records even while paused; evidence gaps are worse than pauses.
+- **Close Guard** (Stop): blocks ending the turn while open `- [ ]` items remain, while any `- [x]` lacks an `-- evidence:` note (evidence-on-close: adjectives don't close cards), while any cited evidence `command` has **no successful run in the evidence log** (machine corroboration: a citation that never ran is not evidence), and — with `REPLAY: on` — while any cited acceptance fails when **re-run now** ('passed once' is not 'still passes').
 - **Profile Injector** (SessionStart): injects tier + routing + habits, **sized to the ledger state** — full when a round is starting/active, minimal when idle, one line when paused.
 
 **Wrap-up lint**: `python3 <skill-dir>/hooks/fable_lint.py <project_dir>` — machine-checks the discipline itself (SPEC source tags present, open cards name acceptance, closed cards carry evidence). Run it at step 7 of the execution template; findings are open work.

hooks/README.md

@@ -4,14 +4,15 @@ The enforcement layer: turn a few of fable-mode's prose rules into Claude Code
 hooks that actually block — ledger-before-delegation and close-verification,
 built around this repo's SPEC.md/PROGRESS.md conventions.
 
-## Four hooks + one lint CLI
+## Five hooks + one lint CLI
 
 | Hook | Event | What it does |
 |---|---|---|
 | `fable_profile_inject.py` | `SessionStart` | When the project has opted in, **auto-inject the tier by model + the six levers + ledger context recovery** (no need to type "use fable mode") |
 | `fable_spawn_guard.py` | `PreToolUse` (Agent\|Task\|Workflow) | When opted in: **block a detailed spawn with no ledger** (forces the plan gate) and **block any spawn requesting a model stronger than the session's** (the model ceiling) |
 | `fable_fail_streak.py` | `PostToolUse` (Bash) | Advisory, never blocks: at every 3rd **consecutive failing command**, inject the attribution ladder (harness → deployment → product; fix the class via an invariant). Streak state: `$TMPDIR/fable-mode-sessions/<sid>.fails`, reset on success. |
-| `fable_close_guard.py` | `Stop` | While the ledger still has unchecked items, **block ending the turn** (cures early stopping / spinning). When all items are checked, **block if any `- [x]` lacks an evidence marker** (`-- evidence:` / `证据:`) — evidence-on-close. |
+| `fable_evidence_log.py` | `PostToolUse` (Bash) | Passive recorder: appends every command's **real outcome** (command, exit code, output tail) to `.fable/evidence.jsonl` — the machine-written record citations are checked against. Records even while PAUSED. |
+| `fable_close_guard.py` | `Stop` | While the ledger still has unchecked items, **block ending the turn** (cures early stopping / spinning). When all items are checked: **block if any `- [x]` lacks an evidence marker** (`-- evidence:` / `证据:`), **block if a cited evidence `command` has no successful run in the evidence log** (machine corroboration), and with `REPLAY: on` **block if a cited acceptance fails when re-run now**. |
 
 `fable_lint.py` is **not a hook** — a one-shot CLI (`python3 fable_lint.py <project_dir>`)
 for wrap-up or CI: SPEC exists and carries source tags ([measured]/[inferred]/[not-shown]
@@ -53,6 +54,7 @@ defaults to the conservative tier. This is SessionStart-only info (there is no
 - [x] 2. done -- evidence: pytest 21/21
 - [~] 3. not this round -- deferred: reason
 PAUSED: reason        <- optional line anywhere: suspend enforcement
+REPLAY: on            <- optional: re-run cited acceptances at turn-end
 ROUTING: frugal       <- optional: model-routing profile for this round
 TIER: throughput      <- optional: concurrency tier for this round
 ```
@@ -141,7 +143,10 @@ use your actual absolute clone path if it differs:
       "command": "python3 ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills/fable-mode/hooks/fable_spawn_guard.py"}]}],
   "PostToolUse": [{"matcher": "Bash",
     "hooks": [{"type": "command",
-      "command": "python3 ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills/fable-mode/hooks/fable_fail_streak.py"}]}],
+      "command": "python3 ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills/fable-mode/hooks/fable_fail_streak.py"}]},
+    {"matcher": "Bash",
+     "hooks": [{"type": "command",
+      "command": "python3 ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills/fable-mode/hooks/fable_evidence_log.py"}]}],
   "Stop": [{"hooks": [{"type": "command",
     "command": "python3 ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills/fable-mode/hooks/fable_close_guard.py"}]}]
 }
@@ -160,6 +165,7 @@ To disable entirely, remove the hooks block from settings.json.
 No third-party deps, just run:
 
 ```bash
-python3 tests/test_guards.py   # 13 cases: opt-in detection, ledger presence, small-spawn/fork exemptions, git-root boundary, loop-safety, fail-open
-python3 tests/test_inject.py   #  9 cases: per-model tier, env override, ledger context recovery, JSON envelope, fail-open
+python3 tests/test_guards.py    # opt-in detection, ledger presence, exemptions, git-root boundary, loop-safety, fail-open
+python3 tests/test_inject.py    # per-model tier, env override, ledger context recovery, JSON envelope, fail-open
+python3 tests/test_evidence.py  # evidence log recording, citation corroboration, REPLAY re-runs
 ```

hooks/_fable_common.py

@@ -125,6 +125,128 @@ def load_session_model(session_id):
 EVIDENCE_RE = re.compile(r"(evidence|verified|证据|凭证|验证)\s*[:：]", re.IGNORECASE)
 
 
+# --- machine-written evidence log (.fable/evidence.jsonl) ---
+#
+# The Evidence Logger hook appends one JSON line per Bash command:
+#   {"ts": <epoch>, "cmd": <command>, "exit": <int>, "tail": <output tail>}
+# The Close Guard checks cited `commands` on `- [x]` cards against this log,
+# so "the acceptance actually ran" is machine truth, not a self-reported note.
+
+EVIDENCE_LOG = "evidence.jsonl"
+EVIDENCE_LOG_MAX_BYTES = 512 * 1024  # rotate: keep the newest half beyond this
+EVIDENCE_TAIL_CHARS = 200
+
+_BACKTICK_RE = re.compile(r"`([^`]+)`")
+
+
+def evidence_log_path(fable_dir):
+    return os.path.join(fable_dir, EVIDENCE_LOG)
+
+
+def response_exit_code(tool_response):
+    """Best-effort exit code from a Bash tool_response; None when unknown."""
+    r = tool_response
+    if isinstance(r, str):
+        m = re.search(r"[Ee]xit code[: ]+([0-9]+)", r)
+        return int(m.group(1)) if m else None
+    if not isinstance(r, dict):
+        return None
+    for key in ("exitCode", "exit_code", "code", "returncode"):
+        v = r.get(key)
+        if isinstance(v, int):
+            return v
+    for key in ("is_error", "isError"):
+        if r.get(key) is True:
+            return 1
+    text = " ".join(str(r.get(k, "")) for k in ("stdout", "stderr", "output"))
+    m = re.search(r"[Ee]xit code[: ]+([0-9]+)", text)
+    return int(m.group(1)) if m else None
+
+
+def append_evidence(fable_dir, cmd, exit_code, tail):
+    """Append one run record; rotate the log when it grows too large.
+    Best-effort, fail-open — recording must never disturb the session."""
+    try:
+        path = evidence_log_path(fable_dir)
+        try:
+            if os.path.getsize(path) > EVIDENCE_LOG_MAX_BYTES:
+                with open(path, encoding="utf-8", errors="replace") as fh:
+                    lines = fh.readlines()
+                with open(path, "w", encoding="utf-8") as fh:
+                    fh.writelines(lines[len(lines) // 2:])
+        except OSError:
+            pass
+        rec = {"ts": time.time(), "cmd": str(cmd)[:2000],
+               "exit": exit_code,
+               "tail": str(tail or "")[-EVIDENCE_TAIL_CHARS:]}
+        with open(path, "a", encoding="utf-8") as fh:
+            fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
+    except Exception:
+        pass
+
+
+def _norm_cmd(s):
+    return re.sub(r"\s+", " ", str(s)).strip()
+
+
+def cited_commands(card_line):
+    """Backtick-quoted commands in the *evidence part* of a `- [x]` line.
+    Returns [] when the evidence note cites no command (prose-only note)."""
+    m = EVIDENCE_RE.search(card_line)
+    if not m:
+        return []
+    return [_norm_cmd(c) for c in _BACKTICK_RE.findall(card_line[m.end():])
+            if _norm_cmd(c)]
+
+
+def evidence_log_has_run(log_path, cited, want_success=True):
+    """True if the log records a run whose command matches `cited`
+    (normalized substring, either direction) — successful when want_success."""
+    try:
+        with open(log_path, encoding="utf-8", errors="replace") as fh:
+            for line in fh:
+                try:
+                    rec = json.loads(line)
+                except ValueError:
+                    continue
+                cmd = _norm_cmd(rec.get("cmd", ""))
+                if not cmd:
+                    continue
+                if cited in cmd or cmd in cited:
+                    if not want_success or rec.get("exit") == 0:
+                        return True
+    except Exception:
+        return False
+    return False
+
+
+def uncorroborated_citations(ledger_p, log_path):
+    """`- [x]` cards whose cited evidence command never ran successfully.
+
+    Machine check for "the acceptance actually ran": a card that cites a
+    `command` as evidence must have a successful run of that command in the
+    evidence log. Cards with prose-only evidence are not checked here (the
+    substantive-string rule still applies to them). Returns [] when the log
+    doesn't exist yet (projects predating the logger) — fail-open.
+    """
+    if not os.path.isfile(log_path):
+        return []
+    bad = []
+    try:
+        with open(ledger_p, encoding="utf-8", errors="replace") as fh:
+            for line in fh:
+                s = line.strip()
+                if s[:5].lower() != "- [x]":
+                    continue
+                cites = cited_commands(s)
+                if cites and not any(
+                        evidence_log_has_run(log_path, c) for c in cites):
+                    bad.append(s)
+    except Exception:
+        return []
+    return bad
+
+
 # --- model-routing profiles (quality / balanced / frugal) ---
 
 ROUTING_PROFILES = ("quality", "balanced", "frugal")
@@ -151,6 +273,22 @@ def read_tier(path):
     return None
 
 
+_REPLAY_RE = re.compile(r"^REPLAY\s*[:：]\s*(on|off)\b", re.IGNORECASE)
+
+
+def read_replay(path):
+    """True when the ledger opts into acceptance replay (`REPLAY: on`)."""
+    try:
+        with open(path, "r", encoding="utf-8", errors="replace") as fh:
+            for line in fh:
+                m = _REPLAY_RE.match(line.strip())
+                if m:
+                    return m.group(1).lower() == "on"
+    except Exception:
+        return False
+    return False
+
+
 def read_routing(path):
     """Per-round routing profile from a `ROUTING: <profile>` ledger line.
 

hooks/fable_close_guard.py

@@ -22,10 +22,66 @@
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 from _fable_common import (  # noqa: E402
     read_hook_input, start_dir, find_fable_dir, ledger_path, parse_ledger,
-    closed_without_evidence,
+    closed_without_evidence, evidence_log_path, uncorroborated_citations,
+    read_replay, cited_commands,
 )
 
 MAX_LIST = 12
+REPLAY_CMD_TIMEOUT = 30   # seconds per acceptance command (FABLE_REPLAY_TIMEOUT)
+REPLAY_TOTAL_BUDGET = 120  # seconds across all replays in one stop
+
+
+def replay_failures(ledger_p, project_root):
+    """Re-run each `- [x]` card's cited acceptance command; list the failures.
+
+    Only runs when the ledger has `REPLAY: on` (checked by the caller) — an
+    explicit opt-in, because re-running acceptances at every stop costs real
+    time. A command that exits non-zero or times out is a failure: 'it passed
+    once' is not 'it still passes'. Budgeted so a heavy suite can't hang the
+    stop indefinitely. Fail-open on unexpected errors.
+    """
+    import subprocess
+    try:
+        timeout = int(os.environ.get("FABLE_REPLAY_TIMEOUT",
+                                     str(REPLAY_CMD_TIMEOUT)))
+    except ValueError:
+        timeout = REPLAY_CMD_TIMEOUT
+    failures = []
+    seen = set()
+    spent = 0.0
+    try:
+        with open(ledger_p, encoding="utf-8", errors="replace") as fh:
+            lines = [l.strip() for l in fh]
+    except Exception:
+        return []
+    import time as _time
+    for s in lines:
+        if s[:5].lower() != "- [x]":
+            continue
+        for cmd in cited_commands(s):
+            if cmd in seen:
+                continue
+            seen.add(cmd)
+            if spent >= REPLAY_TOTAL_BUDGET:
+                failures.append((cmd, "not replayed: %ds replay budget spent "
+                                 "(raise FABLE_REPLAY_TIMEOUT or drop "
+                                 "REPLAY: on)" % REPLAY_TOTAL_BUDGET))
+                continue
+            t0 = _time.time()
+            try:
+                p = subprocess.run(cmd, shell=True, cwd=project_root,
+                                   capture_output=True, text=True,
+                                   timeout=min(timeout,
+                                               REPLAY_TOTAL_BUDGET - spent))
+                if p.returncode != 0:
+                    tail = (p.stderr or p.stdout or "").strip()[-160:]
+                    failures.append((cmd, "exit %d: %s" % (p.returncode, tail)))
+            except subprocess.TimeoutExpired:
+                failures.append((cmd, "timed out"))
+            except Exception as e:
+                failures.append((cmd, "could not run: %r" % e))
+            spent += _time.time() - t0
+    return failures
 
 
 def main():
@@ -65,6 +121,45 @@ def main():
                 "evidence.\n" % (len(bad), path, lines)
             )
             return 2
+        # Machine corroboration: a card that cites a `command` as evidence
+        # must have a successful run of that command in the evidence log
+        # (written by the Evidence Logger hook, not by the model).
+        unc = uncorroborated_citations(path, evidence_log_path(fable_dir))
+        if unc:
+            shown = unc[:MAX_LIST]
+            lines = "\n".join("    " + it for it in shown)
+            if len(unc) > len(shown):
+                lines += "\n    ... and %d more" % (len(unc) - len(shown))
+            sys.stderr.write(
+                "[fable-mode] BLOCKED stop: %d checked card(s) cite an "
+                "evidence `command` with NO successful run recorded in the "
+                "evidence log (%s):\n%s\n"
+                "The log is written by the Evidence Logger hook from real "
+                "tool results — a cited command that never ran (or never "
+                "exited 0) is not evidence. Run the acceptance command now, "
+                "or fix the citation to the command that actually ran.\n"
+                % (len(unc), evidence_log_path(fable_dir), lines)
+            )
+            return 2
+        # Acceptance replay (opt-in via `REPLAY: on`): 'passed once' is not
+        # 'still passes' — re-run each card's cited acceptance before the
+        # round may end, so a later card can't silently break an earlier one.
+        if read_replay(path):
+            fails = replay_failures(path, os.path.dirname(fable_dir))
+            if fails:
+                shown = fails[:MAX_LIST]
+                lines = "\n".join("    `%s` -> %s" % f for f in shown)
+                if len(fails) > len(shown):
+                    lines += "\n    ... and %d more" % (len(fails) - len(shown))
+                sys.stderr.write(
+                    "[fable-mode] BLOCKED stop: REPLAY is on and %d cited "
+                    "acceptance command(s) do not pass when re-run now:\n%s\n"
+                    "A card whose acceptance no longer passes is not done — "
+                    "fix the regression (or, if the command is genuinely "
+                    "stale, fix the citation), then stop.\n"
+                    % (len(fails), lines)
+                )
+                return 2
         return 0  # all closed, all evidenced -> allow stop
 
     shown = open_items[:MAX_LIST]