Freeze and publish v0.4.0: the expanded-breadth release on four analyzers - #74
Merged
Conversation
All 42 normalized reports regenerated at one fixture revision over the expanded populations: Bifrost v0.10.6 (build 18d09c57) across the pinned 118-case smoke slice and thirteen kernels, CodeQL 2.26.3 across eleven, Joern 4.0.610 across six, and Semgrep CE 1.174.0 across eleven. Bifrost v0.10.6 headlines: the smoke slice decides 115/118 (Ruby's pair and the modeled-external calibration are the only non-decisive results); the expanded kernels carry zero decisive mismatches outside Java (37/38) and Kotlin (25/28), with the challenge strata beginning to decide (Java, JavaScript, Python, and TypeScript each gain challenge decisions). The element-object internal_invariant crash reproduces in five languages and Ruby remains fully inconclusive — both tracked upstream (#2637, #2639). CodeQL's first challenge-tier evidence spans all eleven covered languages: 509/626 overall with zero incomplete outcomes; challenge strata score 14-21 of 26 (strong on containers and depth, consistently missing dynamic dispatch). Joern and Semgrep reproduce their retained distributions case-for-case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All forty-two normalized reports — Bifrost v0.10.6 smoke plus thirteen kernels, CodeQL 2.26.3 eleven kernels, Joern 4.0.610 six kernels, and Semgrep CE 1.174.0 eleven kernels — bound at one fixture revision over the expanded challenge-tier populations under the v0.4.0 tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ships the publication layer for the already-committed v0.4.0 freeze (manifest 91b0008a…, revision 306211a, tag v0.4.0): forty-two reports covering Bifrost v0.10.6, CodeQL 2.26.3, Joern 4.0.610, and Semgrep CE 1.174.0 over the first expanded-breadth core — all thirteen kernels carrying the preregistered challenge-tier templates. - Release notes at docs/releases/v0.4.0.md, following the v0.3.0 format: freeze identity, the forty-two bound reports with digests, per-kernel correct decisions with the challenge stratum shown separately, and the reproduction and immutability contracts. Every number derives from results/results.json and reports/freeze.json, and no expanded-core number is compared with a 16-template v0.3.0 one. - Site: v0.3.0's results model archived under docs/src/data/archive/ with its evidenceRef pinned to release commit cd572c4; new /snapshots/v0-4-0/ pages; /current and the sidebar point at v0.4.0, v0.3.0 listed as archived. - LandingResults picks up Joern and Semgrep from the model without a vendor list edit, but three presentation issues came with four analyzers: cards now state the decided fraction beside the population fraction, so a declared-capability decline no longer reads as a wrong answer; four vendors lay out as a 2×2 grid on the card and a fixed-width grid in the dialog rather than one stretched trailing panel; and the per-language core sizes are enumerated instead of rendered as a misleading range. The hand-written verdict is rewritten for this snapshot. - CITATION.cff bumped to v0.4.0; docs/milestones.md gains a factual expanded-breadth status section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The v0.4.0 freeze re-ran all forty-two reports on Bifrost v0.10.6 (build 18d09c57), CodeQL 2.26.3, Joern 4.0.610, and Semgrep CE 1.174.0 at one fixture revision over the expanded populations, but the Bifrost and CodeQL adapter READMEs still narrated the pre-freeze state. Every number below is derived from the committed reports on this tree, never carried over. - Bifrost: heading and build identity moved to v0.10.6 / 18d09c57, and the seven "pending the v0.4.0 freeze-prep re-run" language deferrals (Python, Kotlin, TypeScript, C#, Go, C, Rust) replaced with the runs' actual outcomes — Python 36/58, Kotlin 25/58, TypeScript 34/58, C# 3/58, Go 14/58, C 2/48, Rust 2/54 — each with its challenge-stratum decisive set and diagnostic split. The Java kernel section now reports 37/58 with its classic half at 31/32, and the smoke section states the published dfb-taint-java-direct-positive instability the same way the release notes do: reached in the smoke population, not-reached in the java kernel, same build and same fixture revision, neither re-run to agreement. A C++ paragraph is added for the one kernel that had no evidence prose. The closing paragraph no longer claims v0.10.2-era smoke inconclusives that this build decides. - CodeQL: the nine deferrals replaced with the expanded results (JavaScript, TypeScript, Python, Java 48/58; C# 47/58; Kotlin 46/58; Go 45/58; Rust 44/54; C++ 42/56; C 41/48), each split classic/challenge with its stratum A-D breakdown; the JavaScript configuration hash and the Rust and Ruby timings corrected against the reports; and the contradictory duplicate C++ selection paragraph (56 in one sentence, 32 in the next) collapsed. - Joern and Semgrep reproduced identically, so they only needed the version-context fix: both narrated a per-language table of wave-era fixture revisions, which the whole-adapter re-run replaced with the single v0.4.0 revision 13a11ff4. - Upstream trackers cross-referenced where the prose discusses the crashes and gaps: bifrost-dev #2637 (Ruby), #2638 (Rust gap_contract), #2639 (element-object), #2640 (nested-callable roots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ruby's coverage gap is bifrost-dev #2637; the element-object crash is #2639 and the Rust gap_contract crash is #2638. The notes had the first two swapped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Freezes and publishes v0.4.0 — the first expanded-breadth snapshot: all 13 core kernels at their preregistered challenge-tier denominators, all 42 normalized reports re-run at one fixture revision (
13a11ff4…) on Bifrost v0.10.6 (build18d09c57), CodeQL 2.26.3, Joern 4.0.610, Semgrep CE 1.174.0. 744 frozen cases, 2452 scored case results, revision pinned to thev0.4.0tag.The saturation story is over — nobody is perfect on the expanded kernels:
element-objectcrash (#2639), Rustgap_contract(#2638), Ruby's fully-inconclusive kernel (#2637) — plus an observed instability (java-direct-positivereached in smoke, not-reached in the kernel at the same build) published as observed, not re-run to agreement.deep-relay-chainandrecursive-carry22/22, dynamic dispatch the systematic miss (all 16 reflective-invocation positives missed).Publication layer:
docs/releases/v0.4.0.md, CITATION → v0.4.0, site snapshot with v0.3.0 archived (evidence pinned tocd572c4),/current→ v0-4-0, landing page verified with four vendors (2×2 vendor grids, Semgrep's bounded profile annotated so 14-of-58 reads as declared scope, dialog panels fixed at four-up). Adapter READMEs renarrated against the fresh reports — every "pending the v0.4.0 re-run" deferral replaced with the measured outcome, several stale hashes/timings corrected, and the release-notes issue mapping fixed (#2637 Ruby / #2638 gap_contract / #2639 element-object).Validation
cargo fmt --check·cargo test(106) ·validate(744) ·validate-reports(42) ·validate-freeze·generate-results --check·npm run check(0) ·npm run build(26 pages;/currentredirect and archived v0.3.0 evidence links spot-checked against the pinned commit).🤖 Generated with Claude Code