Freeze and publish DataFlowBench v0.5.0 - #91
Merged
Conversation
Full sequential re-run (bifrost, semgrep, joern, codeql phases) of the 852-case population: 20 Bifrost reports, 17 Semgrep, 12 Joern, 17 CodeQL. First committed evidence for Bifrost v0.10.7 (build 44d9a5be), including the Amendment A9 category-Z modeling cells and the A10-corrected native rationales, with run identities witnessed from the invoked binaries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st modeling and tool-native tiers Release notes derived entirely from results/results.json and reports/freeze.json: Bifrost runner errors 22 -> 0, kernel decisive-correct 222 -> 435, smoke 115 -> 117 of 118, Ruby decides for the first time (0/58 -> 21/58). Four new false positives on v0.10.7 published rather than filtered, and the Java direct-propagation instability restated as still unreconciled. First publication of the modeling matrix (12 templates / 6 categories x 3 languages x 4 adapters) and the tool-native probe set (6 templates x 3 languages x 4 adapters). The site gains tier-aware sections for both, kept in their own populations: no combined leaderboard, benchmark-controlled and tool-native never pooled or visually conflated, model profiles labelled on every matrix row, and declined tiers rendered as coverage rather than as zeros. Vendor colours keep the fixed identity-keyed mapping. Archives v0.4.0's results model pinned at 86a29e0, adds the v0.5.0 snapshot pages, and repoints /current. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The v0.5.0 release: the Bifrost v0.10.7 fix cycle measured, and the first publication of the modeling (#15) and tool-native (#16) tiers. Four commits: the 66-report re-run at one fixture revision, the freeze manifest, the audited results, and the release notes + site snapshot.
Freeze
852 cases / 66 reports / 4 analyzers — Bifrost v0.10.7 (build
44d9a5be, identity witnessed per PR #87), CodeQL 2.26.3, Joern 4.0.610, Semgrep CE 1.174.0 — fixture revisionsha256:9df209ed…, tagv0.5.0at main tip7d688c6(already pushed).validate-freezegreen; the v0.4.0 manifest is superseded, never mutated.Fix-cycle headlines (derived from the frozen results, diffed against v0.4.0's)
gap_contractcrash class (bifrost-dev#2638): 12 runner errors → 10 correct + 2 inconclusive.element-objectcrash cases resolve to inconclusive, not correct; the Javadirect-propagationsmoke-vs-kernel instability persists unreconciled on this build; inconclusive concentrations remain (Ruby 36, Kotlin 30, PHP/C++ 26, C#/TS 24).First tier publications
0/N; a Model profile column keeps benchmark-controlled and tool-native visually distinct; vendor colors keep the fixed identity map.Site
v0.4.0 archived with evidenceRef pinned to
86a29e0; v0.5.0 snapshot pages added;/currentredirects to v0-5-0;astro checkclean, 31 pages build.Verification
cargo fmt --check·cargo test162 ·validate852 ·validate-reports66 ·validate-freezegreen ·generate-results --checkcurrent ·astro check+ build clean.🤖 Generated with Claude Code