diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 6cbb4a2..ebb80e2 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "agent-polygraph", - "version": "0.4.0", + "version": "0.4.1", "description": "Sends Claude back once when its final message says tests pass but the session shows no passing run after the last edit. Adds /agent-polygraph:check for a report on your past sessions. Runs locally, nothing is uploaded.", "author": { "name": "Eleftherios Chondros", diff --git a/README.md b/README.md index 0357103..770ff92 100644 --- a/README.md +++ b/README.md @@ -2,20 +2,41 @@ [![test](https://github.com/EleChon-pixel/agent-polygraph/actions/workflows/test.yml/badge.svg)](https://github.com/EleChon-pixel/agent-polygraph/actions/workflows/test.yml) -Your coding agent said "all tests pass". Did a test pass after its last edit? +Check whether your coding agent's test claims have evidence—and optionally run the tests before it stops. -agent-polygraph is an audit of your past sessions. It reads the Claude Code and Codex session logs already on your machine and checks every "tests pass" claim against what ran: was there a passing test run after the agent's last code edit? It tells you how often your agents claimed something they hadn't verified, and shows the evidence for each case. No setup, no API key, nothing uploaded. +agent-polygraph reads Claude Code and Codex session logs on your machine. Its audit checks detected passing-test claims against the available runs and edits, then shows the evidence behind each verdict. No API key or setup is needed for the default audit, and it uploads nothing. -The audit and guard **read evidence; they do not execute tests**. A flagged claim is not proof the code is wrong: it means the available evidence does not support the claim at that point. If you want automatic test execution, use a separate test-running hook or CI. +```sh +npx --yes agent-polygraph # audit your own sessions +npx --yes agent-polygraph --demo # generated sample sessions +``` + +The audit does not execute tests. The optional guard offers three policies: + +| Policy | Executes tests? | What happens at Stop | +|---|---|---| +| `claims` (default) | No | Challenges definitely unbacked passing-test claims once; allows ambiguous evidence. | +| `strict` | No | Also challenges inconclusive passing-test claims; surfaces unresolved repeated claims. | +| `verify` | Yes | Runs your configured project command regardless of claims, retests repairs, and reports unresolved verification after a bounded repair budget. | + +A flagged claim means the available evidence does not support it at that point. It does not establish that the code is broken or the agent intended to mislead you. A passing verification establishes only that your chosen command passed on the observed inputs. + +To enable executable verification, use a persistent installation and run this in the project you want checked: ```sh -npx agent-polygraph # your own sessions -npx agent-polygraph --demo # generated sample sessions +npm install -g agent-polygraph +agent-polygraph guard --install --mode verify --test-command "npm test" +agent-polygraph guard --status ``` +Replace `npm test` with the command that checks your project. Verification requires a Git checkout for complete fingerprints. Existing installations and the Claude plugin keep the default claim-only policy until explicitly reconfigured. See [guard policies](#guard-policies-optional) for timeouts, repair budgets and caching. + > **Beta / evidence limitations:** The parser has been evaluated on the author's own sessions and public Claude Code traces ([evaluation](docs/evaluation.md)). Controlled replays and real executed tests exist, but **independent, blinded human-validated accuracy has not been established**. If you report a wrong verdict, [open an issue](https://github.com/EleChon-pixel/agent-polygraph/issues/new/choose) with a *manually redacted* example—never upload an unreviewed raw session log. -Historical example from the author's sessions, July–October 2026 (not a representative benchmark or a current-user measurement): +
+Historical example: the author's sessions, July–October 2026 + +This is not a representative benchmark or a current-user measurement. ``` agent-polygraph 32 working sessions · Jul 30 – Oct 31, 2026 · 11.0s @@ -45,7 +66,17 @@ Where the time went Under the tables it prints the latest examples: what the agent claimed, the last test command and its result, what was edited afterwards, and the session file, so you can check it yourself. -**What the benchmarks actually show:** agent-polygraph detects stale test claims more reliably than a *simple any-prior-pass rule* in [25 authored replay scenarios](docs/benchmark.md) (14/14 unbacked claims versus 7/14). In [12 instrumented projects using real test executions](docs/live-evaluation.md), its selective guard requested **5** additional tests and detected **3 of 7** failing final suites; rerunning the full suite every time required **12** additional tests and detected **all 7**. That is an efficiency-versus-coverage trade-off, **not** evidence that the guard improves overall defect detection. [Public-session workload projections](docs/benchmark.md) are separate from actual executed tests. The independently labeled human study is [still open](https://github.com/EleChon-pixel/agent-polygraph/issues/3). +
+ +**Controlled policy comparison:** In [12 authored scenarios with actual test executions](docs/live-evaluation.md), seven final suites failed: + +| Policy | Extra test executions | Failing final suites detected | +|---|---:|---:| +| Claim-only guard (`claims`) | 5 | 3/7 | +| Executable guard (`verify`, cache off) | 12 | 7/7 | +| Always rerun the complete suite | 12 | 7/7 | + +Executable verification closes the demonstrated claim-only coverage gap by running the suite, at the same test-invocation cost as always-rerun plus fingerprint/state overhead. This is a controlled regression result, not an independent accuracy estimate or a general speedup. The [25-case replay](docs/benchmark.md) separately checks claim freshness against a simple any-prior-pass rule; [public-session projections](docs/benchmark.md) estimate workload rather than executed test cost. The independently labeled human study is [still open](https://github.com/EleChon-pixel/agent-polygraph/issues/3). **npm distribution:** Run the latest published release with `npx --yes agent-polygraph --demo`. To find the currently published version before pinning one, use `npm view agent-polygraph version`. The GitHub `main` branch may be ahead of npm. Releases use GitHub OIDC trusted publishing and require maintainer 2FA approval before going live. The [automated public-install check](.github/workflows/verify-published-npm.yml) tests the download from npm, not just the repository source. See [Release instructions](docs/releasing.md). @@ -83,13 +114,13 @@ npx agent-polygraph guard --uninstall staged demo: guard stops a coding agent that claims tests pass after editing code, and the rerun fails -`guard` is a Stop hook that runs the same checks on the live session. When the agent's final message says tests pass and the session doesn't back it up, the agent is sent back once: +The default `claims` policy is a Stop hook that runs the audit checks on the live session. When the agent's final message says tests pass and the session doesn't back it up, the agent is sent back once: > agent-polygraph: your final message says "All tests pass.", but you edited calc.js after the last test run. Run the relevant tests now and report the actual result. If they can't be run, say so instead of claiming they pass. -It blocks at most once per stop, looks only at the final message (so a claim the agent already corrected doesn't count), stays quiet when it can't tell, and never fails the turn because of its own errors. Install it globally (`npm i -g agent-polygraph`) so the hook doesn't start `npx` on every stop. +The `claims` policy requests one corrective continuation, looks only at the final message (so a claim the agent already corrected doesn't count), stays quiet when it can't tell, and never fails the turn because of its own errors. Install it globally (`npm i -g agent-polygraph`) so the hook doesn't start `npx` on every stop. -I tested it with both agents in a staged setup: they were told to edit code after a test run and then claim the tests pass (the GIF above is one of these runs). Both were stopped, reran `npm test` and reported the failure. Codex needs one extra step: it only runs hooks you have approved, and only reads a project's `.codex/` folder if you trust it. After installing, open `codex` in the project once, trust the folder and approve the hook. +I tested the claim-only policy with both agents in a staged setup: they were told to edit code after a test run and then claim the tests pass (the GIF above is one of these runs). Both were stopped, reran `npm test` and reported the failure. Codex needs one extra step: it only runs hooks you have approved, and only reads a project's `.codex/` folder if you trust it. After installing, open `codex` in the project once, trust the folder and approve the hook. ### Verify the project even when the agent makes no test claim @@ -112,16 +143,14 @@ The selected command is the coverage contract: choose the full suite if that is For transcript-only checking with visible ambiguity, use `guard --install --mode strict`. It also challenges inconclusive passing-test claims and surfaces unresolved repeated claims. It does not run tests or discover unclaimed failures. The default `--mode claims` retains the original one-cycle, definite-claim-only behavior; the plugin uses that default unless a project install overrides it. -### Do you need it? - -Maybe not. Two simpler setups also work, and they are more thorough: - -- A Stop hook that runs your test suite after every turn and gives the agent the result. This also catches breakage when the agent doesn't claim anything. -- A Stop hook that always asks the agent to double-check its results. When I tried this on small tasks where the agent had edited code after its last run, it reran the tests every time (6 of 6, Claude Haiku). +### Which policy should you use? -The trade-off is cost. Both fire on every stop, so every turn ends with a test run, and the second also adds a model turn, even when nothing changed or nothing was claimed. A test-running hook also has to know each project's test command and setup (a virtualenv, environment variables, a subfolder, a database). The default `claims` policy runs nothing: it reads the session in well under a second and only sends the agent back when the final message claims a pass that no run after the last edit supports (about 6% of claims in my logs), naming the edit that made the run stale. +- Use the **audit** to investigate past sessions without running tests or changing agent behavior. +- Use **`claims`** when you want occasional verification requests and accept that unclaimed failures and ambiguous evidence can pass through. +- Use **`strict`** when you want ambiguous passing-test claims challenged, while retaining transcript-only operation. It still cannot discover unclaimed failures. +- Use **`verify`** when the chosen suite should run before the agent stops, including after repairs. With caching off it incurs test execution on each Stop within the repair budget. It adds local receipts, fingerprint checks and bounded recovery to a test-running hook. -If your suite runs in a few seconds and needs no setup, running it on every stop provides stronger failure coverage. The `verify` policy does that with bounded repairs and local status receipts. The `claims` policy is for when you want fewer rerun requests and accept narrower coverage. `verify` with caching can reduce repeat executions, but reusing a pass assumes unchanged ignored inputs and external dependencies. +A plain Stop hook that runs your suite is also a reasonable choice for a simple project. Neither policy is a better defect detector than the tests it executes. Optional caching can reduce repeated executions of deterministic local suites, but it adds assumptions about unchanged ignored inputs and external services. ### As a Claude Code plugin @@ -133,11 +162,11 @@ In Claude Code you can install the guard as a plugin instead: On Claude Code older than 2.1.275, run `/plugin marketplace add EleChon-pixel/agent-polygraph` first, then `/plugin install agent-polygraph@agent-polygraph`. -The plugin runs the same Stop hook from its own copy, so nothing is downloaded when the agent stops, and it adds `/agent-polygraph:check` for the report (it takes the same options, for example `/agent-polygraph:check --since 30d`). If you also installed the hook with `guard --install`, the plugin's copy stays quiet so the agent isn't sent back twice. The plugin doesn't include `record`; turn that on separately if you want it. +The plugin runs the `claims` Stop hook from its own copy, so nothing is downloaded when the agent stops, and it adds `/agent-polygraph:check` for the report (it takes the same options, for example `/agent-polygraph:check --since 30d`). If you also installed the hook with `guard --install`, the plugin's copy stays quiet so the agent isn't sent back twice. The plugin doesn't include `record`; turn that on separately if you want it. ## Evidence the logs don't have -Some things never show up in the logs: tests you ran yourself in another terminal, CI results, and files changed by a script that builds their names at run time. Two optional extras cover them. +Some things never show up in the logs: tests you ran yourself in another terminal, CI results, and files changed by a script that builds their names at run time. Optional recording and GitHub CI evidence can reduce those gaps for retrospective analysis. Executable verification provides a separate current test run; its status receipts are not imported into the retrospective claim report. ```sh npx agent-polygraph record --install # this project; add --global for all of them @@ -153,7 +182,7 @@ npx agent-polygraph --ci # GitHub Actions (online) The report shows how many claims each of these settled. -## What it checks +## What the retrospective audit checks | Finding | How it's decided | |---|---| @@ -173,22 +202,22 @@ Background runs are followed: Claude Code's `run_in_background` results (the not ## How it decides -Everything is deterministic pattern matching over the logs. There is no LLM involved, and the same logs always give the same numbers. When a rule is unsure, it says "couldn't tell" instead of flagging the agent. +Retrospective verdicts use deterministic pattern matching, without an LLM. The same transcripts and supplemental inputs produce the same analysis; persisted output files, recorder evidence and online CI results can change those inputs. Ambiguous audit evidence is reported as "couldn't tell". Guard handling of uncertainty depends on its selected policy. - Each command's start and its result are separate events, in the order they were observed. A claim can only be backed by a result the agent had already seen. A run covers the code as it was when the run started, so an edit during the run makes it stale. - In a Codex cell that runs several commands, each command gets its own output. If outputs can't be matched to commands, the result stays unknown. - An edit counts once it happened: a Claude Code edit after its tool result confirms it, a Codex patch if it applied. Shell writes count too (`sed -i`, redirects, `tee`, `cp`/`mv`, scripts that write named files), placed by the `cd` in effect where they appear. - A script's writes are the files it clearly writes, such as `open('x.py', 'w')`, `Path('x.py').write_text(...)`, `writeFileSync('x.js', ...)`, or a variable holding such a path that is written later. Files it only reads don't count. A write to a file it can't name (a loop over paths, `extractall`, `shutil.copy` to a variable) makes the claim "couldn't tell". -- An edit in one checkout or worktree doesn't make a run in another one stale. Only edits to source files count; docs, config, data, generated output and hidden tool folders don't. +- In the retrospective audit, an edit in one checkout or worktree does not make a run in another stale. Only recognized source-file edits count; docs, config, data, generated output and hidden tool folders are excluded. Executable verification separately fingerprints all enumerated non-ignored working-tree files, including configuration and data. - `tail pytest.log` is evidence about the run that wrote `pytest.log`, not a new run. - Results come from the runner's summary line first (`3 failed, 10 passed in 1.2s`, `Tests: 1 failed`, `test result: FAILED`). An exit code is trusted only when the test ran last (`cd app && pytest`, `ruff check .; pytest`) or a plain `&&` chain exited 0. `npm test; true` exits 0 because of `true`. - Commands refused by a permission prompt, the auto-mode classifier or a hook never ran, so they aren't test runs. - Every detected claim sentence in a message is checked separately. The guard quotes a claim that actually lacks evidence, even when another claim in the same message is backed. - Runs in an unrelated checkout do not back the current project. Subagent runs retain their own working directory; ambiguous project scope is left unknown. - A claim is a sentence like "all tests pass" or "62 passed". Plans ("I'll make sure the tests pass"), conditions ("once tests pass"), history ("the latest recorded run passed"), questions, partial results ("58 of 59 pass") and quoted runner output are ignored. English is supported, plus simple forms in Greek, Spanish, Portuguese, German, French and Italian. -- Subagent transcripts are merged into the parent session, so a subagent's passing run backs the parent's claim. +- Subagent transcripts are merged into the parent session. A subagent's passing run can supply evidence when its directory and scope are relevant; an unrelated worktree is not proof of a pass in the parent checkout. - Tests the agent created in the same session can't be "weakened". A Claude Code `Write` over an existing test is an overwrite, so it can. -- `guard` judges only the final message. "Correction: I did not run the tests" after an earlier claim doesn't trigger a block. +- Transcript-only guard policies judge the final message. "Correction: I did not run the tests" after an earlier claim does not trigger a claim-based block. Executable verification runs its configured command independently of the final wording. The rules are in [`src/classify.js`](src/classify.js) and the tests in [`test/`](test), including a labelled set of claim and non-claim sentences ([`test/claims.corpus.test.js`](test/claims.corpus.test.js)). A false positive or a missed claim makes a good [issue](https://github.com/EleChon-pixel/agent-polygraph/issues/new/choose); include the redacted sentence and it goes into that set. @@ -201,20 +230,20 @@ The public Claude Code evaluation reviewed 26 parsed sessions and identified 24 Two reproducible controlled experiments test narrower propositions: - [Stop-hook replay](docs/benchmark.md): 25 authored Claude Code/Codex scenarios. Agent-polygraph correctly blocked 14/14 labeled unbacked claims versus 7/14 for a naive “any test passed earlier” hook, with zero false blocks in these selected cases. -- [Executed-test policy comparison](docs/live-evaluation.md): 12 authored scenarios with actual Node.js test executions. The selective guard required five extra test executions and detected three of seven failing final suites; running the full suite after every response required 12 executions and detected all seven. +- [Executed-test policy comparison](docs/live-evaluation.md): 12 authored scenarios with actual Node.js test executions. The claim-only guard required five extra test executions and detected three of seven failing final suites. Both executable verification with caching off and always-rerun required 12 executions and detected all seven. -The replay labels and scenarios were constructed by the project authors. The guard cannot discover failures the agent never claimed away, or infer whether a narrow passing test establishes a full-suite pass. **An independent, blinded human evaluation is still needed.** The [review issue](https://github.com/EleChon-pixel/agent-polygraph/issues/3) describes the protocol; `scripts/blind-review.mjs` prepares prediction-free review packets. No raw private session logs should be shared with reviewers or issues without permission and redaction. +The replay labels and scenarios were constructed by the project authors. Transcript-only guard policies cannot discover failures without a relevant passing-test claim. Executable verification runs the chosen command regardless of claims, but its coverage depends on that command and the tests themselves. **An independent, blinded human evaluation is still needed.** The [review issue](https://github.com/EleChon-pixel/agent-polygraph/issues/3) describes the protocol; `scripts/blind-review.mjs` prepares prediction-free review packets. No raw private session logs should be shared with reviewers or issues without permission and redaction. ## Limits -These are the gaps that remain. `record` and `--ci` can reduce some of them, but neither replaces running tests or provides independent proof of correctness. +The retrospective audit depends on recorded evidence; `record` and `--ci` can reduce gaps. Executable verification instead runs a current command, with the fingerprint, timeout and caching limits described above. None of these establishes overall code correctness. - Tests outside the logs. Tests the agent ran, test runs Claude Code recorded from hooks, commands you typed with `!` in Claude Code, and test output you pasted into the chat all count. So do your own bash and zsh runs with `record --shell`, and GitHub Actions runs on pushed commits with `--ci`. Other shells (fish, PowerShell, an IDE's test button), other CI services, and CI runs on commits the session never printed are still invisible. - "The full suite passes" prefers an explicitly recognized full-suite command in the same folder, even when no count was printed. If the only observed run is targeted and there is no full-suite evidence, the result is **inconclusive**, not backed. Count-based heuristics still help when the runner cannot reliably identify the suite's scope. A passing test run is evidence of that run, never proof that all code works. - Output with no summary. Codex keeps each command's full output. Claude Code may save long output to a separate file that the analyzer reads if it still exists. New `record` entries retain **parsed test outcomes and pass counts, not raw test output**. A run abandoned before printing a result leaves nothing to read. - Files a script names at run time. Without `record` these make a claim "couldn't tell", never "backed". `record` settles them, but only in git checkouts and only for sessions after it was installed. - New agent versions can change the log format. Supported: Claude Code 2.x (Bash and PowerShell tools; Linux, macOS and Windows paths) and Codex CLI. The report lists the versions it read, warns about newer ones, and counts files it couldn't parse instead of dropping them silently. -- Unbacked is not the same as false. "Code edited after the last run" means the claim wasn't verified after the edit; the edit may have been harmless, such as a comment or a test file. The report can't tell whether the tests would have passed, because nobody ran them at that point. +- Unbacked is not the same as false. "Code edited after the last run" means the claim wasn't verified after the edit; the edit may have been harmless, such as a comment or a test file. The retrospective report cannot establish whether the tests would have passed at that point. A later successful verification does not retroactively establish what evidence the agent had when it made the original claim. - The numbers describe your sessions, projects and prompts. They are not a benchmark of the models. ## Options @@ -230,6 +259,7 @@ These are the gaps that remain. `record` and `--ci` can reduce some of them, but --json full results as JSON: findings, unclear reasons, unreadable files (keep it local) explain investigate flagged/uncertain verdicts (--limit, --since, --session, --project) doctor check local hooks, transcript availability and recorder permissions +guard --status inspect local executable-verification receipts --ci also use GitHub Actions results (goes online, through your gh login) --claude-dir Claude Code projects dir (default ~/.claude/projects, or $CLAUDE_CONFIG_DIR/projects) --codex-dir Codex sessions dir (default ~/.codex/sessions, or $CODEX_HOME/sessions) @@ -252,17 +282,18 @@ Before distributing a new version, compare `npm view agent-polygraph version` wi ## Privacy -New recordings intentionally minimize local data. The agent hook stores a keyed command fingerprint, the runner's result and optional fingerprints of source files. It does **not** save raw command text, test output or copies of repository files in its snapshot store. The shell integration saves only a recognized test-runner category, time, folder and exit code. Local record files, key and snapshot metadata are owner-only on Unix. Hook settings and shell startup files are replaced atomically, preserving their permissions and creating private recovery backups under `~/.agent-polygraph/backups` (see [recovery](docs/releasing.md#recovery-for-installed-hooks)). The secret fingerprint key is stored in `$AGENT_POLYGRAPH_HOME/record-key` (default `~/.agent-polygraph/record-key`). +New recordings intentionally minimize local data. The agent hook stores a keyed command fingerprint, the runner's result and optional fingerprints of source files. It does **not** save raw command text, test output or copies of repository files in its snapshot store. The shell integration saves a recognized test-runner category, time, folder, exit code and full/unknown scope classification. Executable verification saves hashed identifiers, a working-tree/environment fingerprint, outcomes, timestamps and execution/attempt counts under `~/.agent-polygraph/guard`; it stores no raw command, test output or source contents in those receipts. The approved executable command is stored in the installed hook configuration. Local record files, key and snapshot metadata are owner-only on Unix. Hook settings and shell startup files are replaced atomically, preserving their permissions and creating private recovery backups under `~/.agent-polygraph/backups` (see [recovery](docs/releasing.md#recovery-for-installed-hooks)). The secret fingerprint key is stored in `$AGENT_POLYGRAPH_HOME/record-key` (default `~/.agent-polygraph/record-key`). -**Existing installations:** records made before this update may still include full commands, and older versions may have created a local Git object store with duplicate source contents. Upgrading does **not** remove those files. After reviewing whether you still need the historical evidence, uninstall recording and manually remove the old `~/.agent-polygraph/records` and `~/.agent-polygraph/objects` directories. Removing historical records means those sessions will lose their supplemental evidence; the original Claude Code/Codex transcripts remain untouched. Do not delete `record-key` while retaining version 2 records that rely on matching fingerprints. +**Legacy recordings:** older recorder formats may include full commands, and earlier versions may have created a local Git object store with duplicate source contents. Upgrading does **not** remove those files. After reviewing whether you still need the historical evidence, uninstall recording and manually remove the old `~/.agent-polygraph/records` and `~/.agent-polygraph/objects` directories. Removing historical records means those sessions will lose their supplemental evidence; the original Claude Code/Codex transcripts remain untouched. Do not delete `record-key` while retaining version 2 records that rely on matching fingerprints. Stored fingerprints are **not** a guarantee of anonymity: recording metadata includes timestamps, working directories and relative source-file paths, and existing v1 records may still contain raw commands. The original Claude Code/Codex transcripts can also contain secrets and confidential data. The terminal report and `--json` may quote transcript text and paths; do not share them without review. The `--html` report intentionally excludes excerpts, prompts, commands and paths. The recorder only supports bounded source-file snapshots in Git checkouts and skips symbolic links; when evidence cannot be read it must stay inconclusive rather than certifying a pass. New source-file fingerprints include relative filenames and are not an anonymization guarantee. -It reads files and works offline. These operations are opt-in: +The default audit reads local files and works offline. These additional operations are opt-in: - `guard` and `record` edit the hook config files they print when you install or uninstall them, and `record --shell` adds or removes one marked line in `~/.bashrc` and `~/.zshrc`. - `record` saves snapshots and logs in `~/.agent-polygraph`. +- `guard --mode verify` executes your selected command with the hook's normal permissions and writes local verification receipts. That command can itself access the network, modify files or produce artifacts; the guard does not sandbox it. - `--ci` and `explain --ci` contact api.github.com through `gh` and send repository names and date ranges. The terminal report and `--json` include excerpts from your sessions. `--html` doesn't. diff --git a/docs/releases/v0.3.2.md b/docs/releases/v0.3.2.md index 48aa021..ffd1703 100644 --- a/docs/releases/v0.3.2.md +++ b/docs/releases/v0.3.2.md @@ -1,6 +1,6 @@ # agent-polygraph v0.3.2 — claim evidence fixes -Prepared for the staged npm release process. This file does not establish that npm publication has completed. Check `npm view agent-polygraph version` and the public-install verification workflow. +Published on npm and in [GitHub Releases](https://github.com/EleChon-pixel/agent-polygraph/releases/tag/v0.3.2). A clean public npm install was verified on 3 November 2026. The exact npm source commit is `c3cb6c329c5a2621319f74f9ecd2690bb864b33e`. - Check every detected passing-test claim sentence in a message. A backed first sentence no longer hides a later failed claim. The guard quotes the actual flagged sentence. - Abstain when a test ran in an unrelated checkout. Preserve subagent working directories in Claude Code and Codex, and avoid using another project's run for a generic claim in a multi-project session. @@ -10,4 +10,4 @@ Prepared for the staged npm release process. This file does not establish that n Package and Claude plugin versions are both 0.3.2. No dependencies added. Reinstall shell integration with `agent-polygraph record --install --shell` after upgrading to regenerate its copied shell script; existing category-only receipts remain readable with unknown scope. -Validation and the utility assessment are documented in [usefulness-review.md](../usefulness-review.md). Publication requires green main-branch CI, GitHub OIDC staging and maintainer npm 2FA approval. A later GitHub release tag must point to the exact staged source commit. +Validation and the utility assessment are documented in [usefulness-review.md](../usefulness-review.md). The release used [GitHub OIDC staging](https://github.com/EleChon-pixel/agent-polygraph/actions/runs/41689551085) and maintainer npm 2FA approval. The GitHub release tag identifies the exact npm source commit. diff --git a/docs/releases/v0.4.0.md b/docs/releases/v0.4.0.md index 2b7c21e..3b87a3a 100644 --- a/docs/releases/v0.4.0.md +++ b/docs/releases/v0.4.0.md @@ -1,6 +1,6 @@ # agent-polygraph v0.4.0 — executable guard verification -Prepared for OIDC staged publishing. This document does not establish public npm availability. Check the registry and public-install verification after maintainer approval. +Published on npm and in [GitHub Releases](https://github.com/EleChon-pixel/agent-polygraph/releases/tag/v0.4.0). The [public-install verification](https://github.com/EleChon-pixel/agent-polygraph/actions/runs/41697406992) passed. The exact npm source commit is `6c529ebd2717f51f5a71a30c03a7862d3033a875`; later documentation changes do not modify its immutable tarball. - Add `guard --install --mode verify --test-command "npm test"`: execute an explicitly selected command in the fixed project folder at Stop, including stops without any passing-test claim and continued stops after repairs. - Default to fresh execution without pass caching. Add optional content/environment-bound, expiring pass reuse for deterministic local suites. diff --git a/docs/releases/v0.4.1.md b/docs/releases/v0.4.1.md new file mode 100644 index 0000000..ab9a7af --- /dev/null +++ b/docs/releases/v0.4.1.md @@ -0,0 +1,12 @@ +# agent-polygraph v0.4.1 — README correction + +Prepared for the existing OIDC staged-release process. Check npm and the public-install workflow before treating this version as published. + +- Correct the claim that no guard policy executes tests. Distinguish the retrospective audit, transcript-only `claims`/`strict` policies and executable `verify` policy. +- Put installation and executable-verification setup near the top. Explain that existing installations and the plugin retain the default claims policy. +- Include executable verification in the controlled benchmark comparison, with its actual 12-run cost and the limits of authored scenarios. +- Correct statements about unclaimed failures, fingerprints, repeatability and privacy so their scope matches the implementation. +- Collapse the historical example and replace outdated policy-selection advice. +- Update publication records for the already-published 0.3.2 and 0.4.0 releases without moving their tags or changing their tarballs. + +Only documentation and matching package/plugin version metadata change. Runtime behavior, dependencies and hook policy defaults are unchanged from 0.4.0. The new version is necessary because npm's archived 0.4.0 README cannot be overwritten. diff --git a/docs/releasing.md b/docs/releasing.md index 5eaae97..176f215 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -1,6 +1,6 @@ # Publishing agent-polygraph to npm -**Latest public release verified during development:** `agent-polygraph@0.3.2`. A clean public npm install on 3 November 2026 verified its CLI, demo, README, `explain`, `doctor` and integrity metadata after [OIDC staging](https://github.com/EleChon-pixel/agent-polygraph/actions/runs/41689551085). Published versions are immutable. The source now prepares 0.4.0; GitHub commits alone do not publish npm releases. See [v0.4.0 notes](releases/v0.4.0.md). +**Latest public release verified:** `agent-polygraph@0.4.0`, installed and checked from npm on 3 November 2026. Its [GitHub Release](https://github.com/EleChon-pixel/agent-polygraph/releases/tag/v0.4.0) and npm `gitHead` identify source commit `6c529ebd2717f51f5a71a30c03a7862d3033a875`. The [public-install check](https://github.com/EleChon-pixel/agent-polygraph/actions/runs/41697406992) passed after [OIDC staging](https://github.com/EleChon-pixel/agent-polygraph/actions/runs/41695697316). Published versions are immutable. The source now prepares the README-only 0.4.1 patch; GitHub commits alone do not publish npm releases. See [v0.4.1 notes](releases/v0.4.1.md). ## Published v0.3.0 @@ -45,7 +45,7 @@ node scripts/release-check.mjs The archived `0.3.0` tarball cannot be modified in place. The published `0.3.1` patch corrects outdated version and feature-availability language in the README without changing runtime behavior. The public npm install and the corrected README were verified; see [v0.3.1 release notes](releases/v0.3.1.md). -## Prepared evidence patch (v0.3.2) +## Published evidence patch (v0.3.2) Version 0.3.2 was published and verified from the public npm registry on 3 November 2026. See [v0.3.2 release notes](releases/v0.3.2.md) and the [utility review](usefulness-review.md). Its exact npm source commit is `c3cb6c329c5a2621319f74f9ecd2690bb864b33e`; later source changes do not alter that tarball. Users of shell integration should regenerate the copied script after upgrading with `agent-polygraph record --install --shell`. diff --git a/package.json b/package.json index 497076a..4a9e792 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "agent-polygraph", - "version": "0.4.0", + "version": "0.4.1", "description": "Check your coding agents' \"tests pass\" claims against what actually ran. Reads Claude Code and Codex session logs locally.", "type": "module", "bin": {