feat: add fixmap verify and release v0.7.3 (#99)

plan answers where to start. verify answers whether the change that followed
matches the plan, by comparing a saved report against a real git diff.

Five checks, all comparisons between two things the user already has, so
nothing is executed and no repository code runs:

- edits in generated or retired locations, which the next build discards
- files the change needed that the plan never ranked
- a leading file left untouched
- source moving with no test moving, pointing at the routed test
- risk areas the change reached that the plan never flagged

Only the first exits non-zero. That one is wrong regardless of what the task
was; the rest are advisory, because a plan can be wrong and a change can
still be right, and only someone reading both can say which. The output
reports the gap and leaves that judgement alone.

Also fixes test routes carrying a repository-wide related-files list. Every
route received the same set, so a report claimed npm --prefix packages/core
run test would exercise another package's tests. Related files are scoped to
the route's package; a root script keeps everything.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: aryamthecodebreaker

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 All notable changes to FixMap are documented here.
 
+## 0.7.3 - 2026-07-26
+
+### Added
+
+- `fixmap verify --report <file> --diff <spec>` closes the loop after an edit. It compares a saved plan against the diff that followed and reports five things: edits in generated or retired locations that the next build discards, files the change needed that the plan never ranked, an untouched leading file, source moving with no test moving, and risk areas the plan never flagged. Both inputs are things the user already has, so nothing is executed and no repository code runs. Only a discarded edit exits non-zero — that one is wrong regardless of the task, while everything else is advisory, because a plan can be wrong and a change can still be right. Available as `verifyPlan` from the core package and with `--format json`.
+
+### Fixed
+
+- Test routes list only the tests each command can actually run. `findRelatedTests` ran once and its result was assigned to every route, so a report claimed `npm --prefix packages/core run test` would exercise `packages/action/test/runner.test.ts`, which that command never reaches. On this repository all three routes carried an identical eight files spanning three packages. Related files are now scoped to the route's package directory; a repository-root script keeps everything.
+
 ## 0.7.2 - 2026-07-26
 
 ### Added

README.md

@@ -97,6 +97,36 @@ or path from this file in the task to raise it.
 
 It distinguishes the cases that actually differ: the file was ranked, it scored below the cutoff, it was deliberately excluded (a test, a lockfile, generated output whose source was ranked instead), or the scan never saw it. When a scan hits its file limit, it says so rather than implying the path does not exist. Add `--format json` for the machine-readable form.
 
+### Verify the change afterwards
+
+`plan` answers where to start. `verify` answers whether the change that followed matches the plan — by comparing the saved report against a real git diff:
+
+```bash
+npx -y @aryam/fixmap@latest plan --issue "password reset emails fail" \
+  --format json --output fixmap-report.json
+
+# ...make the change...
+
+npx -y @aryam/fixmap@latest verify --report fixmap-report.json --diff main...HEAD
+```
+
+```text
+FixMap verified 3 changed files against the plan and raised 1 error and 2 warnings.
+
+- **error** A file was edited in a generated or retired location. A build regenerates
+  these, so the change will be lost. Edit the source they are produced from.
+  - `dist/auth/reset-password.js`
+- **warning** One file changed that the plan did not rank. Either the task grew beyond
+  the original description, or the ranking missed them — worth checking which.
+  - `src/billing/charge.ts`
+- **warning** Code changed but no test did. The plan routed this test as most related.
+  - `test/reset-password.test.ts`
+```
+
+It checks five things: edits in generated or retired locations, files the change needed that the plan never ranked, an untouched leading file, source moving with no test moving, and risk areas the plan never flagged. Nothing is executed — both inputs are things you already have.
+
+Only a discarded edit exits non-zero, because that one is wrong regardless of the task. Everything else is advisory: a plan can be wrong and a change can still be right, and FixMap reports the gap rather than judging it. `verify` is CLI-only for now; MCP and Action support follow.
+
 ### MCP server
 
 FixMap exposes one stdio tool, `fixmap_plan`, so an agent can request the same report directly.
@@ -167,7 +197,7 @@ jobs:
         with:
           fetch-depth: 0
       - id: fixmap
-        uses: aryamthecodebreaker/FixMap@v0.7.2
+        uses: aryamthecodebreaker/FixMap@v0.7.3
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
 ```
@@ -247,11 +277,11 @@ Read the full [benchmark methodology and scanner measurements](docs/BENCHMARKS.m
 npm run evaluate:heldout
 ```
 
-## What changed in v0.7.2
+## What changed in v0.7.3
 
-`--explain <path>` answers the question a ranked list cannot: why the file you expected is missing. It separates being ranked, scoring below the cutoff, being deliberately excluded, and never being scanned.
+`fixmap verify` closes the loop: it compares a saved plan against the diff that followed, flagging edits a build will discard, files the change needed that the plan never ranked, source moving without tests, and risk the plan never mentioned.
 
-Every published rate now carries a confidence interval, because at twelve cases one result flipping moves Top-3 by eight points. And both evaluations report how often a wrong file ranks *first* while the right one sits lower — the regression suite's 100% Top-3 was concealing a 40% misleading rate, which is what an agent actually pays for.
+Test routes now list only the tests each command can actually run. Every route previously carried the same repository-wide list, so a report claimed `npm --prefix packages/core run test` would exercise another package's tests.
 
 [Inspect the changelog](CHANGELOG.md) · [See the held-out results](benchmarks/heldout/README.md) · [See every regression ranking](benchmarks/external/README.md) · [Audit the efficiency assumptions](docs/BENCHMARKS.md)
 

apps/web/package.json

@@ -15,7 +15,7 @@
     "typecheck": "tsc --noEmit"
   },
   "dependencies": {
-    "@aryam/fixmap-core": "0.7.2",
+    "@aryam/fixmap-core": "0.7.3",
     "next": "16.2.11",
     "react": "19.2.7",
     "react-dom": "19.2.7"

docs/LAUNCH_KIT.md

@@ -207,7 +207,7 @@ Marketplace: https://github.com/marketplace/actions/fixmap
 
 npm: https://www.npmjs.com/package/@aryam/fixmap
 
-Release: https://github.com/aryamthecodebreaker/FixMap/releases/tag/v0.7.2
+Release: https://github.com/aryamthecodebreaker/FixMap/releases/tag/v0.7.3
 
 I would especially value feedback on the ranking explanations and which repository signals would make FixMap more useful before an agent starts editing.
 

package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "fixmap-workspace",
-  "version": "0.7.2",
+  "version": "0.7.3",
   "private": true,
   "description": "Local-first repo context for coding agents: paste a GitHub issue URL to get ranked files, test routes, and risks.",
   "license": "MIT",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@fixmap/action",
-  "version": "0.7.2",
+  "version": "0.7.3",
   "description": "GitHub Action wrapper for FixMap pull request reports.",
   "private": true,
   "license": "MIT",
@@ -11,6 +11,6 @@
     "typecheck": "tsc -p tsconfig.json --noEmit"
   },
   "dependencies": {
-    "@aryam/fixmap-core": "0.7.2"
+    "@aryam/fixmap-core": "0.7.3"
   }
 }

packages/action/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@aryam/fixmap",
-  "version": "0.7.2",
+  "version": "0.7.3",
   "mcpName": "io.github.aryamthecodebreaker/fixmap",
   "description": "Local-first CLI and MCP server mapping GitHub issue URLs, tasks, and diffs to ranked files, tests, and risks.",
   "license": "MIT",
@@ -46,7 +46,7 @@
     "typecheck": "tsc -p tsconfig.json --noEmit"
   },
   "dependencies": {
-    "@aryam/fixmap-core": "0.7.2",
+    "@aryam/fixmap-core": "0.7.3",
     "@modelcontextprotocol/sdk": "1.29.0"
   },
   "engines": {

packages/cli/package.json

@@ -5,6 +5,8 @@ import { fileURLToPath } from "node:url";
 import {
   explainFile,
   renderJsonReport,
+  renderVerifyMarkdown,
+  verifyPlan,
   renderMarkdownReport,
   scanRepo,
   type FileExplanation,
@@ -22,6 +24,7 @@ export type CliOptions = {
   format: "markdown" | "json";
   output?: string | undefined;
   explainPath?: string | undefined;
+  reportPath?: string | undefined;
   unknownArgs: string[];
   invalidValues: string[];
 };
@@ -43,10 +46,12 @@ Usage:
   fixmap plan --issue "Fix login" --repo https://github.com/owner/repository
   fixmap plan --diff main...HEAD
   fixmap plan --base main --head HEAD --format json
+  fixmap verify --report fixmap-report.json --diff main...HEAD
   fixmap mcp
 
 Commands:
   plan                Generate a FixMap report for a task or diff
+  verify              Compare a saved report against the diff that followed it
   mcp                 Run FixMap as an MCP server over stdio for AI coding agents
 
 Options:
@@ -58,6 +63,7 @@ Options:
   --format <fmt>      Output format: markdown (default) or json
   --output <file>     Write the report to a file instead of stdout
   --explain <path>    Explain why one file was ranked where it was, or left out
+  --report <file>     Verify command only: the JSON report the change was planned from
   --help, -h          Show this help
   --version, -v       Show the FixMap version
 `;
@@ -88,7 +94,7 @@ export async function runCli(args: string[], dependencies: CliDependencies = {})
   }
 
   const options = parseArgs(args);
-  if (options.command !== "plan") {
+  if (options.command !== "plan" && options.command !== "verify") {
     stderr(`Unknown command: ${options.command || "(none)"}\n\n${USAGE}`);
     return 1;
   }
@@ -104,6 +110,10 @@ export async function runCli(args: string[], dependencies: CliDependencies = {})
     stderr(`${sections.join("\n")}\n\n${USAGE}`);
     return 1;
   }
+  if (options.command === "verify") {
+    return runVerify(options, { stdout, stderr });
+  }
+
   if (!options.issueText && !options.diffSpec && !options.baseRef) {
     stderr("Provide --issue, --diff, or --base/--head so FixMap has a task signal.\n");
     return 1;
@@ -185,6 +195,7 @@ export function parseArgs(args: string[]): CliOptions {
   let format: "markdown" | "json" = "markdown";
   let output: string | undefined;
   let explainPath: string | undefined;
+  let reportPath: string | undefined;
   const unknownArgs: string[] = [];
   const invalidValues: string[] = [];
 
@@ -236,6 +247,10 @@ export function parseArgs(args: string[]): CliOptions {
       } else {
         invalidValues.push(`--format received ${JSON.stringify(value ?? "(missing)")}; expected "markdown" or "json"`);
       }
+    } else if (arg === "--report") {
+      consumeValue();
+      if (value?.trim()) reportPath = value.trim();
+      else invalidValues.push("--report requires a path to a FixMap JSON report");
     } else if (arg === "--explain") {
       consumeValue();
       if (value?.trim()) explainPath = value.trim();
@@ -259,6 +274,7 @@ export function parseArgs(args: string[]): CliOptions {
     format,
     output,
     explainPath,
+    reportPath,
     unknownArgs,
     invalidValues
   };
@@ -271,6 +287,70 @@ function readVersion(): string {
   return packageJson.version;
 }
 
+/**
+ * Compares a saved plan against the diff that followed it. Both inputs are things the
+ * user already has, so nothing is executed and no repository code runs.
+ */
+async function runVerify(
+  options: CliOptions,
+  io: { stdout: (text: string) => void; stderr: (text: string) => void }
+): Promise<number> {
+  if (!options.reportPath) {
+    io.stderr("Provide --report with the JSON report this change was planned from.\n");
+    return 1;
+  }
+  if (!options.diffSpec && !options.baseRef) {
+    io.stderr("Provide --diff or --base/--head so FixMap can see what changed.\n");
+    return 1;
+  }
+  if (/^https?:\/\//i.test(options.repo ?? "")) {
+    io.stderr("verify needs a local checkout; remote mode cannot resolve a diff.\n");
+    return 1;
+  }
+
+  let report: FixMapReport;
+  try {
+    report = JSON.parse(readFileSync(options.reportPath, "utf8")) as FixMapReport;
+  } catch (error) {
+    io.stderr(
+      `Could not read "${options.reportPath}": ${error instanceof Error ? error.message : String(error)}\n` +
+      "Generate one with: fixmap plan --issue \"...\" --format json --output fixmap-report.json\n"
+    );
+    return 1;
+  }
+  if (!Array.isArray(report.contextFiles)) {
+    io.stderr(`"${options.reportPath}" is not a FixMap JSON report: no contextFiles array.\n`);
+    return 1;
+  }
+
+  try {
+    const repo = await scanRepo({
+      repoRoot: options.repo ?? process.cwd(),
+      diffSpec: options.diffSpec,
+      baseRef: options.baseRef,
+      headRef: options.headRef
+    });
+    const unresolvedDiff = repo.diagnostics.find((diagnostic) => diagnostic.code === "diff-unavailable");
+    if (unresolvedDiff) {
+      io.stderr(`${unresolvedDiff.message}\nVerification needs a resolvable diff to compare against.\n`);
+      return 1;
+    }
+
+    const result = verifyPlan(report, repo);
+    io.stdout(
+      options.format === "json"
+        ? `${JSON.stringify(result, null, 2)}\n`
+        : renderVerifyMarkdown(result)
+    );
+    // A generated-location edit is discarded by the next build, so it fails the command
+    // rather than being reported and ignored. Everything else is advisory.
+    return result.findings.some((finding) => finding.severity === "error") ? 1 : 0;
+  } catch (error) {
+    io.stderr(`${error instanceof Error ? error.message : String(error)}\n`);
+    return 1;
+  }
+}
+
 /** Renders one file's explanation. The summary carries the answer; reasons show the working. */
 function renderExplanation(explanation: FileExplanation): string {
   const lines = [`# Why ${explanation.path}`, "", explanation.summary];

packages/cli/src/cli-runner.ts

@@ -1,6 +1,6 @@
 {
   "name": "@aryam/fixmap-core",
-  "version": "0.7.2",
+  "version": "0.7.3",
   "description": "Deterministic local-first repository scanner, context ranker, and report renderer for coding agents.",
   "license": "MIT",
   "repository": {