Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 44 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,58 @@

All notable changes to FixMap are documented here.

Accuracy figures inside a released entry are the numbers measured **at that release** and are
left as written. The current numbers live on the [evidence page](https://usefixmap.vercel.app/evidence),
which is generated from the recorded results rather than transcribed by hand.

## 0.8.2 - 2026-08-02

Closes the audit sweep filed against v0.8.1.

### Fixed

- Windows paths work throughout. `--exclude`, `.fixmapignore` and the Action input normalize `\` to `/`, so a pattern pasted out of Explorer matches instead of silently matching nothing. A symlink or junction beside its target ranks once, as the real file, rather than filling two identically scored rows.
- Manifests saved with a byte order mark or as UTF-16 parse. `Set-Content -Encoding utf8` writes a BOM and `JSON.parse` rejects one, so every script in the file was skipped — surfacing downstream as `no-test-route` rather than as an encoding problem.
- The URLs people actually paste are accepted: a `?query`, a `#fragment`, a `www.` or `api.` host, the `git@github.com:` SSH form, and `file://`. Nothing that changes which resource is fetched moved — other hosts, credentials and ports are still refused.
- An unresolvable `--diff` exits non-zero even when issue text is available. Exiting 0 told every script checking `$?` that the named diff had been applied, and `changedFiles: []` reads the same whether a diff was empty or never resolved.
- `.vue`, `.svelte`, `.java`, `.php`, `.rb`, `.cs`, `.mts` and `.cts` rank. They were scanned but never marked as source, so a Vue app or a Maven tree returned zero context files.
- Test routing finds `test:unit` and `test:ci`, routes a nested Go module with `-C` so the printed command runs, addresses a yarn workspace by name rather than with Yarn 1's removed `--cwd`, and separates test commands from lint and typecheck so validation cannot crowd out the command that runs the tests.
- The Action stops splitting `exclude` inside brace groups, refuses a credentialed issue URL instead of ranking the token as prose, size-guards the pull request comment before GitHub rejects it, and matches comment authors case-insensitively.
- Stemming keeps the letters that make a word: `pass`, `class`, `process`, `status` and `analysis` stay intact, and `passed`/`pass` converge on one stem.

### Added

- `content-unread`, `tracked-paths-absent`, `duplicate-real-path`, `generated-paths-dominant` and `no-related-tests` diagnostics. Each names a situation the report previously left silent — most importantly a source file whose contents were never read but which still ranked on its path, the shape of the miss behind #274.
- `fixmap_explain` accepts `base`, `head`, `workingTree` and `includeUntracked`, so an agent can explain a ranking from the working-tree plan it just ran. `fixmap_doctor` sets `isError` when the install is unhealthy.
- `/robots.txt` on the site, and a Compare stage on the product page.

### Evidence

- **Hit rates are unchanged.** Held-out 7/12 Top-1, 8/12 Top-3, 9/12 Top-5; development regression 11/16, 16/16, 16/16; adversarial 8/8 with a false-confidence rate of 0.0. Every ranking-affecting change was measured against the held-out suite before and after, one at a time rather than as a batch.
- **Confidence became more conservative and better calibrated.** On held-out, the `high` band went from 6 cases to 3, and from 3/6 correct to 2/3. That follows from `scanComplete` now meaning "every candidate was actually read" rather than "the file limit was not reached": a repository with files past the sample ceiling no longer earns a high label it cannot support. `high` is now more accurate than `medium` on held-out, which is the ordering the label is supposed to have.
- **The held-out composition changed in the v0.8.1 follow-up, not here.** `sindresorhus/got` was rotated into the regression suite once its blob permalink had informed a ranking fix, and a replacement was drawn by the documented rule. So 8/12 Top-3 is not comparable with the 9/12 published before v0.8.1 — different repositories, not a regression.
- **Two proposed ranking fixes were measured and rejected.** A tie-break on definition evidence (#282) dropped held-out Top-1 from 7/12 to 6/12 and did not fix the case it was filed about; reverted, and the issue closed with the numbers. Removing `.css`/`.json` from the source set (#347) measured as an exact no-op on both suites, so nothing shipped and the issue stays open pending a benchmark case that can score it.

### Installation

```bash
npm install --global @aryam/fixmap@0.8.2
fixmap doctor
fixmap --version
```

The package, MCP Registry entry, GitHub tag/release, Action tag, and production site must all resolve to 0.8.2 before the release is considered complete.

## 0.8.1 - 2026-08-01

### Fixed

- Closed the v0.8.0 dogfood backlog across CLI validation, working-tree verification, explain path normalization, comparison output, exclusions, ranking confidence, risk evidence, JSON rank fields, test routing, and Action comment selection.
- Added case-insensitive formats, command-specific help, actionable output-path errors, canonical GitHub URL validation, and clean remote/working-tree conflict errors.
- Added browser-safe compare/exclusion exports and a live Compare/limit/exclude demo surface.

### Added

- Case-insensitive formats, command-specific help, actionable output-path errors, canonical GitHub URL validation, and clean remote/working-tree conflict errors.
- Browser-safe compare/exclusion exports and a live Compare/limit/exclude demo surface.
- MCP parity tools `fixmap_compare` and `fixmap_doctor`, plus working-tree and limit controls.
- GitHub Action inputs for `limit`, `exclude`, `working-tree`, and `include-untracked`; JSON comments now preserve JSON.
- Release gates that verify npm `latest`, canonical homepage metadata, exact internal versions, and a fresh global installation before publishing the GitHub release.
Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Give FixMap a public GitHub issue. It fetches the task, infers the repository, s
npx -y @aryam/fixmap@latest plan --issue https://github.com/chalk/chalk/issues/624
```

No clone, signup, configuration, or source upload is required. Requires Node.js 20.11 or newer.
No clone, signup, configuration, or source upload is required. The CLI requires Node.js 20.11 or newer. The GitHub Action declares `using: node24`, which is the runtime GitHub supplies on its runners and places no requirement on your own Node version.

### Installing

Expand Down Expand Up @@ -58,6 +58,8 @@ npx fixmap --version
npx fixmap plan --issue "password reset emails fail"
```

If `npm install` inside a *source checkout* of this repository fails with `ENOTEMPTY` on a phosphor-icons path, a previous install left a partial directory behind: delete `node_modules` and rerun. This affects contributors building from source on Windows, never anyone installing the published package.

If `cd` or `Set-Location` fails, do not run the project-scoped `npm install` yet: PowerShell
stays in the previous directory, so npm will install there. Run `Get-Location`, create or
select the intended project directory, and then install.
Expand All @@ -71,7 +73,7 @@ npx -y @aryam/fixmap@latest doctor
Remove the stale copy with `npm uninstall -g @aryam/fixmap`, or use the unambiguous form:

```bash
npm exec --yes --package=@aryam/fixmap@0.8.1 -- fixmap --version
npm exec --yes --package=@aryam/fixmap@0.8.2 -- fixmap --version
```

| Command | Answers |
Expand Down Expand Up @@ -272,8 +274,8 @@ npx -y @aryam/fixmap@latest doctor
```text
# FixMap Doctor

- ok Running version: 0.8.1
- PROBLEM Global install: 0.3.1 (this process is 0.8.1)
- ok Running version: 0.8.2
- PROBLEM Global install: 0.3.1 (this process is 0.8.2)
A globally installed fixmap shadows the version npx was asked for. Run
`npm uninstall -g @aryam/fixmap`, or invoke the exact version with
`npm exec --package=@aryam/fixmap@<version> -- fixmap <command>`.
Expand All @@ -282,7 +284,7 @@ npx -y @aryam/fixmap@latest doctor

It exits non-zero when it finds a shadow, so a CI step fails rather than reading on.

Doctor can compare the running package, the first `fixmap` shim on `PATH`, and npm's global package. It cannot infer a version you intended in some other shell command or inspect every historical npm-exec cache entry; when reproducibility matters, use `npm exec --yes --package=@aryam/fixmap@0.8.1 -- fixmap --version` and confirm the printed version before continuing.
Doctor can compare the running package, the first `fixmap` shim on `PATH`, and npm's global package. It cannot infer a version you intended in some other shell command or inspect every historical npm-exec cache entry; when reproducibility matters, use `npm exec --yes --package=@aryam/fixmap@0.8.2 -- fixmap --version` and confirm the printed version before continuing.

### MCP server

Expand Down Expand Up @@ -356,7 +358,7 @@ jobs:
with:
fetch-depth: 0
- id: fixmap
uses: aryamthecodebreaker/FixMap@v0.8.1
uses: aryamthecodebreaker/FixMap@v0.8.2
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
```
Expand All @@ -367,7 +369,7 @@ To close the plan→edit→verify loop without leaving GitHub, save the plan as

```yaml
- id: plan
uses: aryamthecodebreaker/FixMap@v0.8.1
uses: aryamthecodebreaker/FixMap@v0.8.2
with:
format: json
- run: echo '${{ steps.plan.outputs.report }}' > fixmap-plan.json
Expand All @@ -377,7 +379,7 @@ To close the plan→edit→verify loop without leaving GitHub, save the plan as
path: fixmap-plan.json

# In a later run, after the fix is pushed:
- uses: aryamthecodebreaker/FixMap@v0.8.1
- uses: aryamthecodebreaker/FixMap@v0.8.2
with:
mode: verify
report-path: fixmap-plan.json
Expand Down Expand Up @@ -460,9 +462,9 @@ Read the full [benchmark methodology and scanner measurements](docs/BENCHMARKS.m
npm run evaluate:heldout
```

## What changed in v0.8.1
## What changed in v0.8.2

v0.8.1 resolves the complete 107-issue v0.8.0 dogfood backlog. CLI, MCP, and Action now share working-tree, limit, exclusion, format, and validation behavior where their interfaces support it; MCP adds compare and doctor; JSON ranks are explicit; explain resolves normalized absolute Windows paths; verify findings carry consistent risk paths and severity; and generated evaluation results and presentation docs no longer outrank maintained implementation.
v0.8.2 closes an audit sweep filed against v0.8.1. Windows path handling works throughout — exclusions, symlinks, and manifests saved with a byte order mark. `.vue`, `.svelte`, `.java`, `.php`, `.rb`, `.cs`, `.mts` and `.cts` rank, having previously been scanned but never treated as source. The URLs people actually paste are accepted, and an unresolvable `--diff` now exits non-zero instead of reporting success on a plan that was never diff-aware. New diagnostics name what the report used to leave silent, most importantly a file whose contents were never read but which still ranked on its path. Hit rates are unchanged; confidence is more conservative and better calibrated. Two proposed ranking changes were measured, found not to help, and rejected — the numbers are in the CHANGELOG.

Installation is now a release gate rather than a documentation promise. The publish workflow verifies npm `latest`, canonical package homepages, the CLI's exact core dependency, a clean global install with a real plan, the MCP Registry version, and the source commit before it creates the GitHub release. The website includes the complete install paths, all five MCP tools, and a realistic Plan → edit carefully → Verify agent conversation.

Expand Down
4 changes: 2 additions & 2 deletions apps/web/app/docs/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ export default function DocsPage() {

<section id="plan" className="doc-section"><p className="eyebrow">Plan</p><h2>Find the right place to start.</h2><p>Give FixMap one task source: plain issue text, a task file, stdin, a public GitHub issue URL, or a git diff.</p><CopyCommand command={commands.localTask} /><h3>Public issue URL</h3><CopyCommand command={commands.publicIssue} /><h3>Working-tree or branch diff</h3><CopyCommand command={commands.diff} /><p>Remote repository mode is issue-only. Clone the repository locally when you need <code>--diff</code>, <code>--base</code>, or <code>--head</code>.</p></section>

<section id="explain" className="doc-section"><p className="eyebrow">Explain</p><h2>Ask the missing-file question.</h2><p>Use <code>--explain</code> when you expected a path and it did not appear. The response distinguishes four different situations:</p><ul><li>The file ranked, just lower than expected.</li><li>It scored below the report cutoff.</li><li>It was excluded intentionally, such as a generated file whose source ranked instead.</li><li>The scanner never saw it, including when a scan limit was reached.</li></ul><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "reset links fail" --explain src/auth/token.ts'} /></section>
<section id="explain" className="doc-section"><p className="eyebrow">Explain</p><h2>Ask the missing-file question.</h2><p>Use <code>--explain</code> when you expected a path and it did not appear. The response distinguishes five different situations:</p><ul><li>The file ranked, just lower than expected.</li><li>It scored below the report cutoff.</li><li>It tied for a reported place but fell outside <code>--limit</code>.</li><li>It was excluded intentionally, such as a generated file whose source ranked instead.</li><li>The scanner never saw it, including a scan limit, a sparse checkout, or an unsupported extension.</li></ul><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "reset links fail" --explain src/auth/token.ts'} /></section>

<section id="focus" className="doc-section"><p className="eyebrow">Focus</p><h2>Narrow the map to what matters.</h2><p>Demo pages, marketing copy, and documentation often contain every symptom word a product documents, so they compete with the implementation. FixMap knows about conventions like <code>examples/</code>; it cannot know your repository&rsquo;s own layout.</p><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "reset links fail" --exclude apps/web --limit 3'} /><p>Patterns can also live in a <code>.fixmapignore</code> file at the repository root, one per line. The two combine, and <code>--explain</code> reports an excluded file as excluded, naming the pattern that matched.</p><h3>Map what you are editing now</h3><CopyCommand command={'npx -y @aryam/fixmap@latest plan --working-tree --issue "reset flow"'} /><p>That means staged and unstaged tracked changes against <code>HEAD</code>. Untracked files stay out unless you add <code>--include-untracked</code>, so agent metadata does not rank beside real edits.</p><h3>Measure a better task</h3><p>Refine the wording, re-plan, and see whether the real file moved up:</p><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "TOKEN_TTL_MINUTES is ignored" --compare before.json'} /></section>
<section id="focus" className="doc-section"><p className="eyebrow">Focus</p><h2>Narrow the map to what matters.</h2><p>Demo pages, marketing copy, and documentation often contain every symptom word a product documents, so they compete with the implementation. FixMap knows about conventions like <code>examples/</code>; it cannot know your repository&rsquo;s own layout.</p><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "reset links fail" --exclude apps/web --limit 3'} /><p>Patterns can also live in a <code>.fixmapignore</code> file at the repository root, one per line. The two combine, and <code>--explain</code> reports an excluded file as excluded, naming the pattern that matched.</p><h3>Map what you are editing now</h3><CopyCommand command={'npx -y @aryam/fixmap@latest plan --working-tree --issue "reset flow"'} /><p>That means staged and unstaged tracked changes against <code>HEAD</code>. Untracked files stay out of the <em>change set</em> unless you add <code>--include-untracked</code>, so scratch metadata is not reported as an edit. They remain ranking candidates either way — a file an agent just wrote is usually the most relevant thing in the repository.</p><h3>Measure a better task</h3><p>Refine the wording, re-plan, and see whether the real file moved up:</p><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "TOKEN_TTL_MINUTES is ignored" --compare before.json'} /></section>

<section id="verify" className="doc-section"><p className="eyebrow">Verify</p><h2>Compare the plan with the change.</h2><p>Save a JSON plan before editing, then compare it with the real diff afterwards.</p><CopyCommand command={'npx -y @aryam/fixmap@latest plan --issue "reset links fail" --format json --output fixmap-report.json'} /><CopyCommand command={commands.verify} /><div className="doc-note"><WarningCircle size={22} aria-hidden /><p>Verify does not run tests or judge correctness. Most findings are advisory because the plan can be wrong and the change can still be right.</p></div></section>

Expand Down
8 changes: 6 additions & 2 deletions apps/web/app/get-started/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,20 @@ on:

permissions:
contents: read
issues: write
pull-requests: write

jobs:
fixmap:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: aryamthecodebreaker/FixMap@v${siteStats.version}`}</code></pre>
- id: fixmap
uses: aryamthecodebreaker/FixMap@v${siteStats.version}
with:
github-token: ${"$"}{{ secrets.GITHUB_TOKEN }}`}</code></pre>
<div className="button-row"><a className="button primary" href={marketplaceUrl}>Install from Marketplace <ArrowRight size={18} weight="bold" aria-hidden /></a><a className="text-link" href={`${repoUrl}/blob/main/action.yml`}>View action source</a></div>
</div>
</section>
Expand Down
2 changes: 1 addition & 1 deletion apps/web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@aryam/fixmap-core": "0.8.1",
"@aryam/fixmap-core": "0.8.2",
"@phosphor-icons/react": "^2.1.10",
"next": "16.2.11",
"react": "19.2.7",
Expand Down
32 changes: 16 additions & 16 deletions benchmarks/external/results.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,22 +32,22 @@
"calibration": [
{
"confidence": "high",
"cases": 7,
"top1Correct": 4,
"top1Accuracy": 0.571,
"cases": 3,
"top1Correct": 2,
"top1Accuracy": 0.667,
"interval95": [
0.25,
0.842
0.208,
0.939
]
},
{
"confidence": "medium",
"cases": 7,
"top1Correct": 6,
"top1Accuracy": 0.857,
"cases": 11,
"top1Correct": 8,
"top1Accuracy": 0.727,
"interval95": [
0.487,
0.974
0.434,
0.903
]
},
{
Expand Down Expand Up @@ -95,8 +95,8 @@
"lib/core/AxiosError.js",
"lib/adapters/http.js",
"lib/helpers/HttpStatusCode.js",
"index.d.ts",
"lib/adapters/xhr.js"
"index.d.cts",
"index.d.ts"
],
"topConfidence": "medium",
"top1": true,
Expand Down Expand Up @@ -152,7 +152,7 @@
"packages/zod/src/v4/mini/schemas.ts",
"packages/zod/src/v4/core/parse.ts"
],
"topConfidence": "high",
"topConfidence": "medium",
"top1": true,
"top3": true,
"top5Hit": true
Expand Down Expand Up @@ -189,7 +189,7 @@
"integration/server.js",
"types/request.d.ts"
],
"topConfidence": "high",
"topConfidence": "medium",
"top1": true,
"top3": true,
"top5Hit": true
Expand Down Expand Up @@ -261,7 +261,7 @@
"lib/EnvironmentPlugin.js",
"lib/dependencies/CreateRequireParserPlugin.js"
],
"topConfidence": "high",
"topConfidence": "medium",
"top1": false,
"top3": true,
"top5Hit": true
Expand Down Expand Up @@ -333,7 +333,7 @@
"src/adapter/lambda-edge/handler.ts",
"src/jsx/context.ts"
],
"topConfidence": "high",
"topConfidence": "medium",
"top1": false,
"top3": true,
"top5Hit": true
Expand Down
Loading