Bind each snapshot's profiles page to its own freeze - #154
Merged
Merged
Conversation
`analyzerProfiles()` read `currentSnapshot` from module scope and `AnalyzerProfiles.astro` took no snapshot prop, so every `snapshots/<slug>/profiles/` page rendered the *current* freeze's figures. After #152 this became visible: `v0-6-0/profiles/` and `v0-6-1/profiles/` published v0.7.0's numbers — Bifrost 225 of 369 positive assertions on all three pages, where v0.6.1 published 223 and v0.6.0 219. That contradicts the promise on `/snapshots/`, that "no page below has been rewritten since its freeze". - `analyzerProfiles(snapshot = currentSnapshot)` takes the snapshot as a parameter and derives both `coreKernelPopulations` and `kernelCorpusDistributions` from it. - `AnalyzerProfiles.astro` accepts `snapshotVersion`, defaulting to the current freeze, in the shape `LatencyTier.astro` already carries. - The three `profiles.mdx` pages name their own version. - The component's "every latency caveat on the latency page" link now resolves to the page's own freeze rather than the newest one — the same leak in link form, which sent a reader of the archived v0.6.0 page to v0.7.0's caveats. No frozen evidence, results model, or scored cell changes: the correct figures were already in each snapshot's own results model, and the site was reading the wrong one. ## Verification `npm test` 7 pass · `astro check` 0 errors · 52 pages built. In the built HTML the three pages now differ exactly where they should: Bifrost recall v0.6.0 59.3% (219/369) v0.6.1 60.4% (223/369) v0.7.0 61.0% (225/369) Bifrost precision 98.2% · 100.0% · 100.0% The other seven analyzers' correctness axes are identical across the three, and the eight speed medians are byte-identical on all three pages — which is what v0.6.1's and v0.7.0's own caution blocks assert, and what the shared v0.6.0 latency corpus requires. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
analyzerProfiles()readcurrentSnapshotfrom module scope andAnalyzerProfiles.astrotook no snapshot prop, so everysnapshots/<slug>/profiles/page rendered the current freeze's figures rather than its own.After #152 this became visible in the built site:
dist/snapshots/v0-6-0/profiles/anddist/snapshots/v0-6-1/profiles/carried v0.7.0's numbers — Bifrost 225 of 369 positive assertions on all three pages, where v0.6.1 published 223 and v0.6.0 published 219. That contradicts the promisesnapshots/index.mdxmakes, that "no page below has been rewritten since its freeze".The fix
docs/src/data/profiles.ts—analyzerProfiles(snapshot: Snapshot = currentSnapshot)takes the snapshot as a parameter and derives bothcoreKernelPopulations(snapshot.results)andkernelCorpusDistributions(snapshot)from it. The default keeps the unversioned call site working.docs/src/components/AnalyzerProfiles.astro— accepts asnapshotVersionprop resolved throughsnapshotByVersion, in the shapeLatencyTier.astroalready carries.profiles.mdxpages name their own version.The latency axis is deliberately untouched
profiles.tsreaches latency throughkernelCorpusDistributions, which selects its corpus bysnapshot.latencyEvidenceRelease. v0.6.0, v0.6.1 and v0.7.0 all bind the v0.6.0 corpus, so passing each page its own snapshot leaves all eight speed marks identical across the three — which is exactly the mixed-run caveat v0.6.1's and v0.7.0's pages already carry, now derived rather than coincidental.Verification
npm test7 pass ·astro check0 errors, 0 warnings, 0 hints · 52 pages built. Read back out of the built HTML, the three pages now differ exactly where they should and nowhere else:Bifrost alone moving is what both caution blocks assert — v0.6.1's "the other seven analyzers did not move at all between these freezes", and v0.7.0's "three of the four re-pins moved nothing at all on the correctness axes".
Not an amendment, on my reading — flagging for the maintainer's call
No frozen evidence, results model, manifest, or scored cell changes here. The correct figures were already in each snapshot's own results model and the site was reading the wrong one, so this is a rendering defect rather than an evidentiary correction under the
docs/new-analyzer.mdtaxonomy — noA<n>number taken. It did, however, put wrong figures on two published pages for the life of #152, so if that reads as warranting a dated amendment, say so and I will add one.No regression test
docs/src/data/*.test.tsruns under barenode --test;profiles.ts→snapshots.tspulls in extensionless specifiers and attribute-less JSON imports that only Vite resolves, so the module cannot be loaded there without new machinery. The built-HTML comparison above is the check.🤖 Generated with Claude Code