All notable changes to FixMap are documented here.
Accuracy figures inside a released entry are the numbers measured at that release and are left as written. The current numbers live on the evidence page, which is generated from the recorded results rather than transcribed by hand.
- Every plan now contains a separate Impact Graph built from direct imports, reverse dependents, routed tests, and repeated Git co-change relationships. Each relationship carries its evidence, confidence, and inspection order; generated and backup artifacts stay excluded.
fixmap contextbuilds a deterministic Markdown or JSON source pack from primary and impact files. It selects line ranges within an estimated source-token budget and records roles, reasons, confidence, truncation, and omissions.fixmap graphexports the Impact Graph as portable Mermaid or versioned JSON while preserving relationship direction and evidence.fixmap watch --report plan.json --repo .emits verification and recalculated impact only when the local working tree changes, with Markdown or JSON Lines output.fixmap benchmark --repo . --last 50backtests BM25-over-code, FixMap context, and FixMap with Impact Graph on identical historical parent-snapshot corpora. It reports all, path-mentioned, and unmentioned cohorts, Wilson intervals, raw cases, skip counts, and secondary-file recall without executing repository code or scoring generated twins as primary answers.fixmap plan --format agentemits a compact, stable handoff organized as EDIT CANDIDATE, INSPECT, TEST, RISK, AVOID, and UNCERTAINTY.- A frozen four-arm agent-study protocol and validator are checked in for future controlled measurements. No agent-effectiveness or time-saved claim is made without completed, auditable runs.
- A 32-second motion-first agent comparison is available on the README and website in animated-preview and 1080p H.264/AAC formats, with original no-vocals music and no claim of measured agent efficiency.
- Verify recalculates impact around the files actually changed and adds advisory findings for high-evidence related paths outside the original plan.
- Plan, Context, Graph, MCP, the GitHub Action, slash-command discovery, the live demo, package documentation, and the website expose the new evidence consistently.
- Git-history collection is bounded, cached, non-executing, and explicit about shallow, truncated, missing, or unreadable history. Import and test relationships continue to work when history is unavailable.
- The original held-out benchmark and its honest BM25 comparison remain unchanged. The repository benchmark is a separate local backtest and does not rewrite frozen evidence.
- The release candidate must pass workspace typechecking, all automated tests, lint, dependency audits, production builds, metadata and generated-artifact checks, package and Action smoke tests, evaluation gates, the agent-study protocol check, scanner performance checks, and desktop/mobile browser verification before any release or deployment action.
fixmap setupinstalls a discoverable/fixmapcommand for Claude Code, Cursor, GitHub Copilot, and Agent Skills; invoking it without a task opens the complete feature catalog.fixmap featuresexposes the same catalog directly.fixmap validate <report.json>checks the versioned report contract, while--fail-on warningand the Action'sfail-on: warningprovide an opt-in strict Verify gate.- Plan, Explain, MCP, and the Action expose a visible fresh-scan path with
--no-cache/noCache, and cache hits now report age with bounded expiry and eviction.
- Ranking, grounding, file mentions, definition detection, import proximity, clustering, confidence, risk vocabulary, diff evidence, exact literals, Unicode text, inflections, typographic quotes, and generated/backup/configuration heuristics were corrected across the complete #500-#627 issue sweep.
- Repository mapping now handles nested package-manager lockfiles, conflicting roots, Windows and absolute exclusion paths, large ignore lists, tracked dirty cache states, BOM-less UTF-16 task files, oversized diffs, more source extensions, and language/test routing for C#, Java, PHP, Ruby, Go, Rust, Python, and JavaScript ecosystems.
- Explain, Compare, Verify, Validate, Doctor, remote repository acquisition, CLI parsing/output, MCP schemas, and copy-paste commands now fail clearly and preserve safe paths instead of silently accepting ambiguous or malformed state.
- GitHub Action comments, outputs, and job summaries stay within platform limits; API errors, pagination, permissions, credentials, Markdown, temporary clones, and generated bundle/metadata drift are handled defensively.
- CI is least-privilege, timeout-bounded, non-duplicating, immutable-action pinned, full-tree audited, non-mutating, and configured for Node 20.11/22 plus Windows and macOS compatibility coverage.
- The publish workflow scopes write/OIDC permission to the release job and verifies the exact main commit, version metadata, npm packages, clean global install, MCP Registry entry, and GitHub Release before succeeding.
- The website uses a restrained editorial product system with clearer navigation, evidence, metadata, copy feedback, visibility-safe motion, and an honest interactive demo instead of synthetic marketing presentation.
- README and the npm package documentation now contain the full FixMap feature catalog,
/fixmapsetup, exclusion syntax, cache/report compatibility, Action controls, and public evidence boundaries.
- The release ledger maps all 126 open GitHub issues in #500-#627 (excluding already-closed #542 and #599) plus every item in the attached 30-finding review to a resolution and verification source.
- External, held-out, adversarial, BM25/lexical/path baselines, context-size proxy, scan-performance, rendered examples, package smoke tests, and the 1,000-file scan gate were regenerated and checked. The held-out evidence continues to state plainly where BM25 leads FixMap.
npm install --global @aryam/fixmap@0.8.9
fixmap doctor
fixmap setup
fixmap featuresThe release is complete only when the exact main commit is tagged v0.8.9, npm and MCP Registry serve 0.8.9, the GitHub release and Action tag resolve to that commit, production serves the matching website, and every public smoke check passes.
- Exact git-state repository scan caching now accelerates repeated Plan, Explain, Compare, Verify, MCP, and Action runs. Non-git and untracked-file states stay uncached,
cache-hitmakes reuse visible, and--no-cacheforces a fresh scan (#495). - JSON plans now carry
reportVersion: 1with a documented additive-compatibility policy, and every report consumer shares the same structural validator (#468, #488). - The website now follows the system colour scheme, includes a persistent light/dark toggle and accessible focus colour, and gives every route its own canonical and Open Graph metadata (#485-#487, #491-#492).
- Scanner diagnostics now distinguish oversized, non-text, and unreadable source; identify skipped submodules and resolved empty diffs; preserve paths in Markdown; use consistent decimal kB units; handle odd UTF-16BE manifests; and report friendly non-repository, unborn-repository, and missing-Git failures (#452-#453, #464, #469-#473, #481, #489-#490).
- Explain receives diff content in every interface, Verify rejects plans from a different repository and avoids duplicate generated-artifact findings, and generated counterparts rank below their maintained source and no longer become the next action (#451, #454, #465, #493).
- CLI validation now rejects plan-only
--report, unresolved comparisons, empty evaluation inputs, and accidental leading-@file reads while preserving literal task text; the Action reserves its Markdown truncation fence, validates reports consistently, canonicalizes supported GitHub URL forms, bounds comment pagination, and separates explicit issue inputs from event-derived context (#455, #457-#460, #462-#463, #468, #482, #496). - Checklist-only issues retain their task text, exclusions explain the matching files they removed, deployment ranking no longer treats bare HTTP status numbers as infrastructure terms, Go/Python/type-declaration tests are recognized, and new unplanned risk areas use an explicit warning severity without an unreachable informational branch (#456, #466-#467, #471, #476).
- The live four-stage walkthrough now writes and reuses one
plan.json, quotes the complete task, and uses valid button-group semantics. Its agent transcript and the product illustration are generated from the real FixMap report engine rather than hand-written output (#463, #478-#480, #483, #492). - The homepage and evidence page read the checked-in adversarial record, CI checks rendered assets for drift, the PowerShell first-run project contains rankable source, and the README is a focused 153-line entry point that routes detailed material to maintained docs (#475, #480, #484, #494).
- Added 23 self-evaluation cases covering the first half of the v0.8.8 issue set, and release CI rejects empty evaluation files before calculating rates (#461, #474).
- The release verification matrix for all 46 issues is published in
docs/releases/v0.8.8-issue-verification.md.
npm install --global @aryam/fixmap@0.8.8
fixmap doctor
fixmap plan --issue https://github.com/chalk/chalk/issues/624The npm core and CLI packages, MCP Registry entry, GitHub tag/release, Action tag, README, changelog, recorded evidence, and production website must all resolve to 0.8.8 before the release is considered complete.
- A pretty-printed vendored dependency bundle could rank first at high confidence. It escaped content-based bundle detection — 96 characters per line against a 400-character threshold, and one bundler marker against a threshold of two — and it escaped generated-duplicate filtering, which by design only drops a generated path that has a maintained source twin. A vendored dependency has none, so it fell through both. Detection now accepts a single marker when the path is itself conventional generated output, such as a
compiled/segment (#446).
- Added a pretty-printed vendored-bundle case to the adversarial suite, which was previously all minified fixtures. The suite is 9/9 with a false-confidence rate of 0.0.
- Chalk's
source/vendor/supports-color/index.jsstill ranks first for a colour-detection task. It is a vendored path whose content genuinely is the only implementation of the behavior, and it is the counterexample any path-based rule has to keep passing. - Held-out remains 7/12 Top-1, 8/12 Top-3, 9/12 Top-5; external remains 11/16, 16/16, 16/16.
The fix above was merged to main before this release, and for a short window the published changelog described it as shipped while npm still served 0.8.6 without it. That is the #274 defect with its polarity reversed: the site advertising a fix the released artifact lacks, rather than an accuracy figure it no longer achieves. The website changelog no longer has an unreleased state — every entry on it is a version you can install.
npm install --global @aryam/fixmap@0.8.7
fixmap doctor
fixmap chalk/chalk#624The npm packages, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.7 before the release is considered complete.
- Stylesheets that repeat product symptom words are deprioritized for non-UI implementation tasks, so presentation copy does not outrank the code that owns the behavior. CSS/layout tasks remain unpenalized, and JSON continues through the existing configuration classification (#347).
- An explicitly named generated artifact remains visible when a task is genuinely about that artifact, but a maintained-source twin caps its confidence at medium and adds a reason naming the generated/source relationship. The mention boost no longer turns build output into an unexplained high-confidence edit target (#362, #371).
- Added regression fixtures for both failure modes and their valid counterexamples: a genuine CSS task still ranks its stylesheet, and a task about a stale generated artifact still sees that artifact.
- All 419 workspace tests, typechecking, lint, production builds, production security audit, Action bundle parity, smoke tests, and scanner benchmark pass.
- Held-out remains 7/12 Top-1, 8/12 Top-3, 9/12 Top-5; external remains 11/16, 16/16, 16/16; adversarial remains 8/8 with a false-confidence rate of 0.
npm install --global @aryam/fixmap@0.8.6
fixmap doctor
fixmap plan --issue https://github.com/chalk/chalk/issues/624The npm packages, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.6 before the release is considered complete.
- The README, package guide, and website now lead with a proper global installation followed by the short
fixmap plan ...command. CLI and MCP examples use the installed binary instead of repeating a long npm invocation. - The one-off npx path now states that npm may prefer an existing project-local binary. Doctor's exact-request detector is described accurately: it works when Doctor 0.8.4 or newer starts, while an older binary that wins first cannot contain newer diagnostic code. The printed running version is authoritative (#437).
- Reproduced the remaining boundary from a child project containing FixMap 0.8.3: an exact 0.8.4 npm request started the project-local 0.8.3 binary and that older Doctor reported only what its old code knew.
- Verified the isolated-prefix/direct-shim procedure independently, and verified the installed-command website with web typechecking, lint, and a production build before release validation.
npm install --global @aryam/fixmap@0.8.5
fixmap doctor
fixmap plan --issue https://github.com/chalk/chalk/issues/624The npm packages, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.5 before the release is considered complete.
fixmap doctornow detects when npm requested an exact FixMap version but executed a different local or ancestor installation. It reports both versions and exits non-zero instead of calling the shadowed process healthy (#437).- The reproducible installation path now uses an isolated npm prefix and invokes that prefix's shim directly. The previous npm exec recommendation could itself be redirected to an older ancestor installation on Windows.
- Reproduced the failure against the published v0.8.3 package with a real 0.8.1 ancestor install before changing source.
- Added regression coverage for mismatched, matching, and non-exact npm package requests. All 417 workspace tests, typechecking, lint, production builds, the production security audit, Action bundle parity, smoke tests, held-out/external/adversarial gates, and the scanner benchmark pass.
npm install --global @aryam/fixmap@0.8.4
fixmap doctor
fixmap --versionThe npm packages, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.4 before the release is considered complete.
fixmap_compareover MCP now rejects a truncated{ "contextFiles": [] }object instead of returning a successful unchanged comparison. Complete FixMap reports that legitimately contain zero matches remain valid, and optional rank, score, and confidence fields are type-checked when present (#398).
- Reproduced the failure with a real stdio MCP client against the published
@aryam/fixmap@0.8.2package before changing source. - The MCP regression suite, all 414 workspace tests, typechecking, lint, production builds, Action bundle parity, smoke tests, held-out/external/adversarial gates, scanner benchmark, and production audit pass. Ranking inputs and recorded hit rates are unchanged.
npm install --global @aryam/fixmap@0.8.3
fixmap doctor
fixmap --versionThe package, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.3 before the release is considered complete.
Closes the audit sweep filed against v0.8.1.
- Windows paths work throughout.
--exclude,.fixmapignoreand the Action input normalize\to/, so a pattern pasted out of Explorer matches instead of silently matching nothing. A symlink or junction beside its target ranks once, as the real file, rather than filling two identically scored rows. - Manifests saved with a byte order mark or as UTF-16 parse.
Set-Content -Encoding utf8writes a BOM andJSON.parserejects one, so every script in the file was skipped — surfacing downstream asno-test-routerather than as an encoding problem. - The URLs people actually paste are accepted: a
?query, a#fragment, awww.orapi.host, thegit@github.com:SSH form, andfile://. Nothing that changes which resource is fetched moved — other hosts, credentials and ports are still refused. - An unresolvable
--diffexits non-zero even when issue text is available. Exiting 0 told every script checking$?that the named diff had been applied, andchangedFiles: []reads the same whether a diff was empty or never resolved. .vue,.svelte,.java,.php,.rb,.cs,.mtsand.ctsrank. They were scanned but never marked as source, so a Vue app or a Maven tree returned zero context files.- Test routing finds
test:unitandtest:ci, routes a nested Go module with-Cso the printed command runs, addresses a yarn workspace by name rather than with Yarn 1's removed--cwd, and separates test commands from lint and typecheck so validation cannot crowd out the command that runs the tests. - The Action stops splitting
excludeinside brace groups, refuses a credentialed issue URL instead of ranking the token as prose, size-guards the pull request comment before GitHub rejects it, and matches comment authors case-insensitively. - Stemming keeps the letters that make a word:
pass,class,process,statusandanalysisstay intact, andpassed/passconverge on one stem.
content-unread,tracked-paths-absent,duplicate-real-path,generated-paths-dominantandno-related-testsdiagnostics. Each names a situation the report previously left silent — most importantly a source file whose contents were never read but which still ranked on its path, the shape of the miss behind #274.fixmap_explainacceptsbase,head,workingTreeandincludeUntracked, so an agent can explain a ranking from the working-tree plan it just ran.fixmap_doctorsetsisErrorwhen the install is unhealthy./robots.txton the site, and a Compare stage on the product page.
- Hit rates are unchanged. Held-out 7/12 Top-1, 8/12 Top-3, 9/12 Top-5; development regression 11/16, 16/16, 16/16; adversarial 8/8 with a false-confidence rate of 0.0. Every ranking-affecting change was measured against the held-out suite before and after, one at a time rather than as a batch.
- Confidence became more conservative and better calibrated. On held-out, the
highband went from 6 cases to 3, and from 3/6 correct to 2/3. That follows fromscanCompletenow meaning "every candidate was actually read" rather than "the file limit was not reached": a repository with files past the sample ceiling no longer earns a high label it cannot support.highis now more accurate thanmediumon held-out, which is the ordering the label is supposed to have. - The held-out composition changed in the v0.8.1 follow-up, not here.
sindresorhus/gotwas rotated into the regression suite once its blob permalink had informed a ranking fix, and a replacement was drawn by the documented rule. So 8/12 Top-3 is not comparable with the 9/12 published before v0.8.1 — different repositories, not a regression. - Two proposed ranking fixes were measured and rejected. A tie-break on definition evidence (#282) dropped held-out Top-1 from 7/12 to 6/12 and did not fix the case it was filed about; reverted, and the issue closed with the numbers. Removing
.css/.jsonfrom the source set (#347) measured as an exact no-op on both suites, so nothing shipped and the issue stays open pending a benchmark case that can score it.
npm install --global @aryam/fixmap@0.8.2
fixmap doctor
fixmap --versionThe package, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.2 before the release is considered complete.
- Closed the v0.8.0 dogfood backlog across CLI validation, working-tree verification, explain path normalization, comparison output, exclusions, ranking confidence, risk evidence, JSON rank fields, test routing, and Action comment selection.
- Case-insensitive formats, command-specific help, actionable output-path errors, canonical GitHub URL validation, and clean remote/working-tree conflict errors.
- Browser-safe compare/exclusion exports and a live Compare/limit/exclude demo surface.
- MCP parity tools
fixmap_compareandfixmap_doctor, plus working-tree and limit controls. - GitHub Action inputs for
limit,exclude,working-tree, andinclude-untracked; JSON comments now preserve JSON. - Release gates that verify npm
latest, canonical homepage metadata, exact internal versions, and a fresh global installation before publishing the GitHub release.
npm install --global @aryam/fixmap@0.8.1
fixmap doctor
fixmap --versionThe package, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.1 before the release is considered complete.
Closes all 22 open reports from a dogfooding sweep of v0.7.4.
- Go and Rust repositories now get a test command. Both toolchains have exactly one, so it can be routed rather than guessed at; a
cargo testin a workspace is scoped to the crate holding the ranked file, for the same reason package scripts already are. Python is deliberately not routed — pytest, tox, unittest and nox are all plausible for a repository carrying apyproject.toml, and FixMap cannot read which one it configures, so it names a runner in the diagnostic instead of claiming a command it cannot verify (#153, #134). fixmap doctorreports the running version, the resolved path, any conflicting global install, and the Node version, exiting non-zero on a shadow. This was the first thing that went wrong in the original dogfooding session: a stale global madenpx @aryam/fixmap@0.7.3run 0.3.1, soverifyappeared not to exist in a release that shipped it (#148, following #103).plan --compare previous.jsonprints what changed between two plans — files that entered or left, rank, score and confidence deltas, and whether task grounding changed. This makes FixMap's own feedback loop measurable in one command: add an identifier, watch the definition site rise (#154).--excludeand.fixmapignoreleave named paths out of ranking. FixMap's penalties know about conventions likeexamples/; they cannot know that a particular repository keeps a marketing site whose copy deliberately contains every symptom word the product documents.--explainreports an excluded file as excluded, naming the pattern that matched (#145).--limitcaps how many context files are reported, on the CLI and over MCP. The useful signal is usually the top one to three; the rest burns agent context (#149).--working-treemaps staged and unstaged tracked changes against HEAD, with--include-untrackedas a separate opt-in. Reaching this through--diff HEADworked but swept in untracked files, which on an agent-driven checkout means scratch metadata ranking beside the edit (#147).- Progress phases on stderr for remote clones and scans, when stderr is a TTY or
FIXMAP_PROGRESS=1. A cold clone sat silent for up to ninety seconds and read as hung (#150). fixmap_explainover MCP, so an agent with no shell can ask why a file is missing instead of re-inventing the ranking (#144).- Pull request URLs are accepted as task input. GitHub serves a pull request's title and body from the same endpoint an issue uses, and "map what this PR is about" is a more common starting point than an issue link. Compare, tree, discussion and file URLs stay rejected: they carry no task text to rank against (#146).
- The GitHub Action gains
mode: verifywithreport-path, closing the plan-edit-verify loop for workflows that never touch the CLI or MCP (#151).
- Confidence is scarce again. It came from an absolute score threshold, so a real Zod task labeled all eight results high while the leader was nineteen points clear — teaching an agent that the eighth guess is as safe to edit as the first. High now requires leading, tying the lead within two points, or carrying definition-site evidence; and a leader that merely out-talks a definition site below it is capped at medium, because that competitor has the stronger kind of evidence (#143, addressing #140).
- Language detection reads the root manifest instead of asking whether any scanned file ends in
.py. clap-rs/clap is Rust and keeps one helper script under.github/, and was told to go inspectpyproject.toml(#152). - Diagnostics no longer echo unbounded user text. A pasted blob travelled verbatim into JSON reports, CI logs, and pull request comments — an observed message was 30,155 characters — and a mistyped
--diffdid the same twice over, because git echoes the failing command inside its own error message (#136, #139). - The file-mention pattern no longer backtracks quadratically. Its body run contains
., so it competed with the\.that follows: on an unbroken run with no extension the engine matched everything, failed, and retried one character shorter from every start position. 30,000 characters took 2.4 seconds, on a code path the Action feeds issue text from public pull requests. 120,000 characters now cost 297 ms. Found while fixing the diagnostic above; not separately reported. verifyhonors--output. It was documented, accepted, and silently ignored: output went to stdout and no file was created (#141).- Duplicate
--repo,--format,--diff,--base,--head,--output,--reportand--explainflags now fail instead of silently keeping the last value. Only--issuedid. Silent last-wins is worst for--repo, which then scans a different tree than the one named first, and--format, which hands a consumer a contract it did not ask for (#142). - The verify hint no longer prints
--diff <base>...HEADinside a copy-paste command. v0.7.4 stopped inventingmain...HEADand replaced it with a placeholder that looks runnable and is not; the command is now runnable as printed, with the part the user must supply on its own line outside it (#137). - MCP
fixmap_verifyaccepts its report inline or as a file path, mirroring CLI--report. Requiring the object form rejected agents that had used the CLI first without naming a shape that would work, and forced the model to re-embed an entire plan in the tool call (#138). verify --format jsoncarries scan diagnostics, so both commands hand an agent the same{code, severity, message, paths?}entry shape and consuming either no longer requires branching on output shape (#133).- An empty diff explains itself instead of printing two
- None foundsections that read like a display bug (#135). - The README and the
fixmap_plantool description both said a high-confidence top result is right "about three quarters of the time". The measured figure in the table directly below it is 9/15 — 60%. Three quarters is the medium band. Found while re-checking the calibration numbers.
- All three accuracy suites are unchanged: internal 62.5/87.5, development regression 67/100/100, held-out 58.3/75/75. The adversarial suite remains 8/8 with a false-confidence rate of 0.0.
- Top-1 calibration is unchanged as well (high 9/15, medium 6/8, low 2/4), because the confidence rules almost never disturb a genuine leader. That also means the suites cannot see that change: they score only the top-ranked file, and the fix is about the other seven rows. The evidence for it is measured behavior on the repositories in the reports — Zod and dayjs both go from 8/8 labeled high to 1/8, with the correct file keeping high in each.
- Verified against the named repositories rather than fixtures alone: clap-rs/clap and spf13/cobra for test routing, Zod and dayjs for confidence, and a live pull request URL for the adapter change.
- Definition sites now outweigh vocabulary-dense consumers, regex quantifiers contribute searchable repeated tokens, and example/demo/presentation surfaces receive a meaningful task-aware penalty (#102, #105, #128).
- URL scheme and host text no longer becomes ranking evidence; unsupported standalone GitHub PR, discussion, compare, tree, and file URLs fail clearly; issue URLs are checked against local or remote repository identity (#109, #112, #113).
- Short trailing-e words no longer collapse into three-letter noise stems, and generic improve-the-codebase wording is classified as vague and returns no edit list (#106, #108).
- Homogeneous repositories keep task vocabulary when boilerplate filtering would otherwise remove every query term (#119).
- Empty, vague, low-confidence, clustered, remote, and saved plans now receive accurate next-step guidance. Verify hints preserve explicit repository/diff context and never invent
main...HEAD; empty diffs without task text fail clearly (#104, #110, #111, #121, #127). --explaindistinguishes score ties outside the top-N limit from files that actually scored below the reporting floor (#116).- Tracked generated release artifacts such as
packages/action/dist/index.mjsproduce a rebuild warning instead of a discarded-edit error (#107). - The GitHub Action fetches canonical issue URLs before ranking, rejects cross-repository issue input, and accepts format values case-insensitively while rejecting invalid values (#123, #124).
- MCP trims whitespace-only inputs and rejects them as missing task signals (#125).
- Issue-only risk notes use low severity and prospective wording; full severity is reserved for diff evidence (#126).
- Reports with code context but no supported test runner emit an explicit
no-test-routediagnostic, including Python guidance (#114). - The web demo now describes the measured
sendMailranking rather than implying the transport file wins (#129).
--issue-file <path>,--issue -stdin, and--issue @pathsupport large/private task text without Windows command-line limits; duplicate--issueflags now fail instead of silently using the last value (#115, #117, #118).- MCP now exposes
fixmap_verify, closing the plan-edit-verify loop for MCP-only agents (#120). - The README documents the npm/npx stale-global-shim failure mode and the unambiguous
npm exec --package ... -- fixmapinvocation (#103).
- Refreshed all recorded evaluation outputs after the ranking changes. The untouched held-out suite remains 9/12 Top-3 (75%) and is now 7/12 Top-1 (58%); the development regression suite improves to 10/15 Top-1 (67%) while remaining 15/15 Top-3/5. The adversarial suite remains 8/8 with zero false-confidence cases.
fixmap verify --report <file> --diff <spec>closes the loop after an edit. It compares a saved plan against the diff that followed and reports five things: edits in generated or retired locations that the next build discards, files the change needed that the plan never ranked, an untouched leading file, source moving with no test moving, and risk areas the plan never flagged. Both inputs are things the user already has, so nothing is executed and no repository code runs. Only a discarded edit exits non-zero — that one is wrong regardless of the task, while everything else is advisory, because a plan can be wrong and a change can still be right. Available asverifyPlanfrom the core package and with--format json.
- Test routes list only the tests each command can actually run.
findRelatedTestsran once and its result was assigned to every route, so a report claimednpm --prefix packages/core run testwould exercisepackages/action/test/runner.test.ts, which that command never reaches. On this repository all three routes carried an identical eight files spanning three packages. Related files are now scoped to the route's package directory; a repository-root script keeps everything.
plan --explain <path>answers the question a ranked list cannot: why a file you expected is missing. It separates the cases that actually differ — ranked, scored below the cutoff, deliberately excluded as a test or lockfile or generated output whose source was ranked instead, or never scanned. A scan that hit its file limit says so rather than implying the path does not exist. Available asexplainFilefrom the core package and as--format json.- Every published hit rate now carries a 95% Wilson confidence interval. At twelve cases one result flipping moves Top-3 by eight points, so the point estimates read far more precisely than the evidence supports: held-out Top-3 is 75% with an interval of 47–91%. The README says so rather than quoting two significant figures.
- Both evaluations report a misleading-top-result rate: how often a wrong file ranks first while the right one sits lower in the window. Held-out is 1/12; the regression suite is 6/15, which its 100% Top-3 had been hiding. This is what an agent actually pays for, since it opens the first file.
-
Committed minified bundles no longer compete with the source a task is about. Repositories ship pre-bundled third-party dependencies — Next.js keeps them under
src/compiled/— and because they have no first-party counterpart the generated-duplicate rule keeps them, while their minified text contains the exact symbol names being searched for. A 30 KB single-line bundle ranked at high confidence, one point behind the real implementation, and even earned the definition-site boost. Files averaging 400+ characters per line are now deprioritized. The check reads content rather than paths, so readable vendored source of any length is untouched and chalk'ssource/vendor/supports-color/index.jsstill ranks first. -
Vague-task detection no longer misses a wordy request. It gated on five task tokens or fewer, so "clean this up and make the general performance better overall" was classified as descriptive purely for being longer than "improve DX". Vagueness is now judged by how little survives removing generic-improvement vocabulary, which also keeps a concrete request that merely asks for an improvement out of the bucket. The adversarial suite found this and now asserts the label strictly.
-
A truncated scan says what it did not read. Hitting the file cap reported only that scanning stopped; it now names the busiest unread directories — "3,000 paths went unread, mostly under web/ (3,000)" — so a reader can judge whether the omission touched the code their task is about. Git checkouts get the exact remainder; a plain directory walk has no complete list to report from and keeps the shorter message.
-
Confidence is now calibrated rather than asserted. Both evaluation suites record the top-ranked file's confidence label, and the summary reports how often each label was correct. Across all 27 cases a top result labeled high confidence is the right fixing file 11/15 of the time, medium 5/8, and low 1/4. The ordering holds, so the label carries information — but high means roughly three in four, not certainty, and the counts are published because bands this small cannot support a precise percentage.
-
Added an adversarial suite measuring the opposite failure to accuracy: fabricated identifiers, real identifiers from the wrong repository, vague requests, absent feature surfaces, runtime-only symptoms, and generic-term floods, run against real pinned repositories. False-confidence rate is 0.0 across 8 cases. Unit tests cannot replace this — FixMap's own fixtures contain the fabricated identifiers, so a suite pointed at this repository would resolve them and pass while the behavior was broken.
npm run evaluate:adversarial:gate. -
The
fixmap_planMCP tool description and the README now tell an agent how much to trust the output: check the analysis block, verify identifiers resolved, widen the search when grounding is weak, and never edit a file only because it ranked highly. -
Published example reports for the three ways FixMap declines to answer — fabricated identifiers, a vague request, and terms that match nothing — so the examples directory no longer shows only successful routings. Regenerate with
npm run render:examples. -
Added a held-out evaluation suite: 12 MIT-licensed repositories selected by the same frozen rule as the regression suite, but chosen after the v0.7.1 ranker was finished and never tuned against. It measures 67% Top-1 and 75% Top-3/5, against 60% / 100% / 100% on the regression suite. Top-1 does not degrade on unseen repositories; the Top-3 gap is what fitting bought on the tuned set. Run it with
npm run evaluate:heldout. -
scripts/evaluate-external.mjsaccepts--suite external|heldout, so both suites share one harness and one recorded-result format.
- The README now reports held-out and regression figures side by side and states plainly that the 75% is the number to plan around. The previously advertised 100% Top-3 was measured on the cases that guided development and was never a generalization estimate.
- The benchmark card no longer headlines a "98.6% fewer tokens" context proxy or a "14.97 minutes saved" comparison. Both compared against assumed baselines rather than measurements — no agent reads an entire repository, and no with/without-agent experiment was run — so a reader had no way to tell the honest numbers on the card from the invented ones. Byte-based context proxies remain recorded and labeled in
docs/BENCHMARKS.md. benchmarks/external/is documented as a regression suite rather than an accuracy claim.
- Scanning a git checkout no longer discards first-party source that happens to sit in a conventionally generated directory.
git ls-files --exclude-standardalready applies.gitignore, so re-applying a hardcoded directory blocklist on top only removed files the author had deliberately committed. Asked to routehandle chalk color detection on windows terminals, FixMap missedsource/vendor/supports-color/index.js— chalk's only implementation of that behavior — and reported no diagnostic explaining the omission. It now ranks first. Directory walks outside a git checkout have no.gitignoreto consult and still skip those directories. - A term is treated as repository-wide boilerplate only when at least 85% of files carry it, rather than half. The old cutoff mistook subject matter for boilerplate: chalk names "color" in 55% of its files, so a color-detection task had its only search term suppressed and returned nothing.
- Generated output is excluded from ranking when the source it was built from is present, because the next build overwrites any edit made there. A committed bundle no longer crowds out the module it was produced from. Vendored code with no maintained counterpart stays rankable, and naming a build artifact explicitly still surfaces it.
- Backup directories and tool-left duplicate filenames are deprioritized, so an agent is not routed into a retired copy. A
.bak,conflicted copy, orquarantine/snapshot no longer outranks the file still in use.
- An empty report now explains itself instead of printing "Diagnostics: None found". FixMap distinguishes task text that produced no searchable term from terms that matched no file, and names the terms it searched for.
- Added repository-grounded identifier analysis to reports. Exact, partial, unresolved, and unverified identifiers are distinguished before ranking confidence is assigned.
- Added regressions for paraphrased camelCase identifiers and identifiers beyond the 64 KB text-sampling boundary, preventing grounding from suppressing useful component words or claiming absence from incomplete evidence.
- Expanded the frozen external benchmark from 6 to 15 pinned repositories and added a reproducible scan/runtime/context-size benchmark plus an exact SVG result card.
- Improved the freshly measured 15-repository baseline from 40% / 67% / 67% to 60% / 100% / 100% Top-1/3/5. The baseline was run against the same frozen cases rather than read from the previously stale results file.
- Confidence is capped when identifier grounding is incomplete, task text is vague, the repository scan is incomplete, or the ranking is too flat to justify certainty.
- Ranking now recognizes member references, type-focused tasks, HTTP/2-to-
h2naming, explicit nested paths and repeated literals while filtering unchecked issue-template options. - The efficiency benchmark labels byte-derived token figures as estimates and the manual-triage comparison as an assumption, not a controlled agent experiment.
- Re-recorded
benchmarks/external/results.jsonwith all 15 exact Top-5 rankings.
- Added bounded, explainable definition-site ranking for distinctive task identifiers and exact code or literal fragments, including truncated literals from issue excerpts.
- Added focused regressions for exact literal extraction, generic-identifier noise, and definition-site ranking.
- Improved the frozen six-repository evaluation from 50% / 83% / 83% top-1/3/5 to 67% / 100% / 100%. The unchanged Zod #5944 case now ranks its fixing
regexes.tsfile first.
- Updated Next.js to 16.2.11 and pinned patched Sharp/libvips and
fast-urireleases, clearing all high-severity production audit findings. - Added a production audit gate that fails CI on high or critical advisories. The remaining two moderate findings come from the MCP SDK's unused HTTP static-file adapter; FixMap exposes only the SDK's stdio transport and does not import or run that adapter.
- Shortened the MCP Registry description to its 100-character limit so the official registry publication can complete.
- Added a repository CI check and release preflight validation for MCP server metadata, preventing registry-only constraints from failing after npm packages have already published.
- CLI and MCP users can paste a canonical public GitHub issue URL as the issue input. FixMap anonymously fetches its title and body and infers the matching repository when no repository is supplied.
- A
remote-issue-fetcheddiagnostic records the exact issue source and whether its body was truncated before ranking.
- Issue URL inputs accept only credential-free canonical HTTPS URLs on
github.com; queries, fragments, encoded separators, and mismatched explicit repository URLs are rejected. - Issue content is fetched only from the fixed
api.github.comendpoint without redirects or credentials, with a 15-second timeout, a bounded API response, a 20,000-character body cap, stable rate-limit errors, and explicit pull-request rejection.
- The README, npm package page, MCP tool description, growth kit, and production quick start now lead with the single-input public issue workflow.
- Root GitHub Action metadata enables the shorter
uses: aryamthecodebreaker/FixMap@v0.5.1install path and makes the repository eligible for a Marketplace listing.
- File mentions now bridge JavaScript build paths to their TypeScript source equivalents, so a task naming
core/Ky.jscan ranksource/core/Ky.tsas explicit context. - Import-graph proximity can no longer boost a neighbor above the higher-scoring seed that supplied the evidence.
- Example/demo files and TypeScript declaration files are deprioritized for runtime implementation tasks, while tasks that explicitly target them keep the normal ranking behavior.
- Evaluation failures now print hit rates as real percentages instead of decimal fractions followed by a percent sign.
- The frozen six-repository evaluation now measures 50% top-1, 83% top-3, and 83% top-5, up from 33% / 33% / 67% on v0.5.0. Zod #5944 remains the documented miss.
- Exact per-case external rankings are now checked in, and scheduled/release runs fail if live output drifts from that reviewed artifact.
- Package and MCP descriptions now lead with deterministic local-first analysis and public GitHub URL support.
- Release publishing now gates on the external evaluation and builds the GitHub release body from the matching changelog section.
- One-command public GitHub repository analysis in the CLI and MCP server: pass a canonical
https://github.com/owner/repositoryURL as the repository input and FixMap will scan an anonymous depth-one temporary checkout (#54). - An informational
remote-repo-fetcheddiagnostic records the canonical source URL, default branch, and fetched commit so remote reports remain reproducible.
- Remote inputs accept only credential-free HTTPS URLs on
github.com. Git credential and askpass helpers, inherited Git configuration and tokens, hooks, submodules, symlinks, and LFS smudging are disabled for the temporary checkout. - Temporary checkouts are removed on success, clone failure, or analysis failure. Cleanup failure is a hard error rather than a successful report with source left on disk.
- Remote URL mode is explicitly issue-only; diff analysis continues to require a local checkout with the requested refs.
- Published package metadata now includes homepage, issue tracker, and discovery keywords.
- Release publishing now validates the selected tag and every version field, verifies npm and MCP Registry artifacts before creating the public GitHub release, and includes the MIT license in both npm packages.
- Public copy describes FixMap output as an explainable report rather than claiming checks were executed as a review receipt.
- Official MCP Registry metadata and OIDC publication, allowing MCP directories to discover
io.github.aryamthecodebreaker/fixmap.
- Static JavaScript/TypeScript import-graph proximity: files one or two import edges from a high-confidence context file are boosted with inspectable reasons such as
imported by ranked file <path>(#12). - Gated-test visibility: changed test files always appear in the test route's related files, and env-gated suites (
describe.skipIf(...)) emit agated-test-skippeddiagnostic naming the enabling environment variable (#33). - A pnpm + Turborepo workspace example proving nearest-package script routing, guarded by a report-drift smoke check in CI (#14).
- A deterministic scanner performance benchmark with published baselines in
docs/BENCHMARKS.md; CI asserts scan correctness only, never wall-clock timing (#15). - A reproducible cross-repository ranking evaluation: six real fixed issues in permissively licensed repositories pinned to exact commits, honest top-1/3/5 hit rates, and a weekly workflow (#13).
- A reproducible animated CLI demo at the top of the README, rendered from live CLI output on the checked-in example (#17), plus a desktop screenshot of the live site (#39).
- Marketplace branding metadata for the GitHub Action (#16).
- Files explicitly named in the task text now rank into context files, including test files, with an
explicitly named in the taskreason; JavaScript/TypeScript reserved words no longer count as content matches (#22). - Diff risk severity is grounded in the files actually changed: risk areas matched only by ranked context report low severity with explicit wording instead of a false high-severity claim (#35).
- The GitHub Action stays green when comment permissions are read-only, as on forked pull requests: it warns, keeps the report in the step summary and outputs, and only fails on unexpected errors (#16).
- README versioned prose is now version-neutral with changelog pointers, and the Action example pins the latest release tag (#34).
- A nonexistent
--repopath now fails with a clear error and nonzero exit instead of an empty success report; the MCP tool returns an error result for the same case (#21). - Repository scans respect
.gitignorein git repositories viagit ls-files, so ignored build output such as.vercel/no longer outranks source files;.verceland.netlifyare also hard-ignored in non-git scans (#23). - Common stop words ("not", "does") and stem fragments ("doe") no longer count as content matches, deployment-related tasks now rank root configuration files such as
vercel.jsonandpackage.json, and lockfiles are excluded from context ranking (#22). - An unresolvable diff ref with no
--issuefallback now exits nonzero in the CLI and returns an error from the MCP tool instead of an empty success report (#25). - Working-tree diff specs such as
--diff HEADnow include untracked files inchangedFiles, so brand-new files rank as changed context; commit-to-commit diffs are unchanged (#26).
- MCP server mode:
fixmap mcpruns a Model Context Protocol server over stdio, exposing thefixmap_plantool to Claude Code, Cursor, Windsurf, and other MCP clients. buildFixMapReportin@aryam/fixmap-core: one call from task input to a complete report, now shared by the CLI, the GitHub Action, and the MCP server.
- Publish the CLI and core packages under the verified npm account scope,
@aryam. - Correct install commands and package links without changing v0.2 engine behavior.
- File-kind-aware ranking with confidence levels.
- Workspace-aware npm, pnpm, Yarn, and Bun test routing.
- Visible diagnostics for invalid git diffs, invalid package manifests, and scan limits.
- Machine-readable GitHub Action outputs.
- Checked-in ranking evaluation cases and CI gate.
- Interactive product demo and social preview image.
- Scoped npm package manifests for one-command CLI usage.
- Reduced documentation and configuration noise for code-focused tasks.
- Limited large repository scans and expanded ignored generated directories.
- Upgraded the web app to stable patched Next.js and React releases.
- Replaced the broken Next.js lint command with ESLint.
- First public deterministic CLI and GitHub Action MVP.