release: FixMap v0.8.2 - #432
Merged
Merged
Conversation
Re-records both accuracy suites from this tree. Hit rates are unchanged — held-out 7/12, 8/12, 9/12 and regression 11/16, 16/16, 16/16 — but confidence moved: the high band on held-out went from 6 cases to 3, and from 3/6 correct to 2/3. That follows from scanComplete now meaning "every candidate was actually read" rather than "the file limit was not reached", so a repository with files past the sample ceiling no longer earns a label it cannot support. high is now more accurate than medium on held-out, which is the ordering the label is for. The CHANGELOG gains the Evidence section 0.8.1 shipped without, and states that the held-out composition changed in the 0.8.1 follow-up — so 8/12 is not a regression against the 9/12 published before it, just different repositories. It also records the two ranking proposals that were measured and rejected. The held-out gate named the wrong suite in every failure it reported, which is a poor property for the check standing between a regression and a release. Documentation: the npm README described a plan+verify surface three releases old; the docs page counted four explain statuses when there are five; the get-started Action snippet had drifted from the README on both checkout version and permissions; and the growth, launch-kit and checklist documents now carry a dated banner pointing at the generated evidence page rather than reading as current truth. The stop-word tradeoff and the Windows ENOTEMPTY workaround are written down where they will be found. Versions: root, core, cli, action, server.json and the lockfile move to 0.8.2. apps/web keeps 0.0.0 deliberately — it is unpublished and publish.yml validates only its core dependency. The lockfile was patched by hand; regenerating it on Windows drops the @emnapi entries Linux CI needs (9 present, verified). Closes #280, closes #285, closes #308, closes #316, closes #324, closes #336 Closes #377, closes #386, closes #399, closes #404, closes #405, closes #413 Closes #417 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
B12 of the sweep, plus the version bump. 13 issues.
npm run cigreen; server manifest and Action metadata checks pass.Suites re-recorded — and confidence moved
Hit rates unchanged. Calibration changed, for the better:
The
highband halved in size and became more accurate. That follows fromscanCompletenow meaning "every candidate was actually read" rather than "the file limit was not reached" — a repository with files past the sample ceiling no longer earns a label it cannot support.highis now more accurate thanmediumon held-out, which is the ordering the label is supposed to have.CHANGELOG (#280, #308)
0.8.1 shipped without an Evidence section; 0.8.2 has one. It states plainly that the held-out composition changed during the 0.8.1 follow-up —
gotrotated into regression once it had informed a fix — so 8/12 is not a regression against the 9/12 published before it. It also records the two ranking proposals that were measured and rejected (#282, #347). Two bullets beginning "Added" were sitting under 0.8.1's Fixed heading.A note at the top says accuracy figures inside released entries are historical by design, with the live numbers on the generated evidence page.
The gate named the wrong suite (#316)
evaluate-external.mjs --suite heldoutreported every failure as "External evaluation" — poor behavior for the check standing between a regression and a release. It says "Held-out evaluation" now.Documentation drift
#405 the npm README documented a plan+verify surface three releases old — no
doctor,--explain,--compare,--limit,--exclude, or--working-tree. #417 the docs page counted four explain statuses; there are five (outside-limitwas missing). #404 the get-started Action snippet had drifted from the README on bothactions/checkoutversion and permissions — aligned to the README's known-working set rather than guessing a narrower one. #336 / #399 the growth log, launch kit and launch checklist now carry a dated banner pointing at the generated evidence page. #386 / #324 / #413 the stop-word tradeoff, the WindowsENOTEMPTYworkaround, and the fact thatusing: node24is GitHub's runner runtime rather than a requirement on your Node are all written down now.Version bump
Root, core, cli, action,
server.jsonand the lockfile move to 0.8.2.apps/webkeeps0.0.0deliberately — it is unpublished, andpublish.yml:93validates only its@aryam/fixmap-coredependency, never its version. The lockfile was patched by hand: regenerating it on Windows drops the@emnapientries Linux CI requires (9 present, verified).Closes #280, closes #285, closes #308, closes #316, closes #324, closes #336, closes #377, closes #386, closes #399, closes #404, closes #405, closes #413, closes #417
🤖 Generated with Claude Code