Skip to content

Freeze and publish DataFlowBench v0.5.0 - #91

Merged
DavidBakerEffendi merged 4 commits into
mainfrom
dave/v0.5.0-freeze
Aug 28, 2026
Merged

Freeze and publish DataFlowBench v0.5.0#91
DavidBakerEffendi merged 4 commits into
mainfrom
dave/v0.5.0-freeze

Conversation

@DavidBakerEffendi

Copy link
Copy Markdown
Collaborator

The v0.5.0 release: the Bifrost v0.10.7 fix cycle measured, and the first publication of the modeling (#15) and tool-native (#16) tiers. Four commits: the 66-report re-run at one fixture revision, the freeze manifest, the audited results, and the release notes + site snapshot.

Freeze

852 cases / 66 reports / 4 analyzers — Bifrost v0.10.7 (build 44d9a5be, identity witnessed per PR #87), CodeQL 2.26.3, Joern 4.0.610, Semgrep CE 1.174.0 — fixture revision sha256:9df209ed…, tag v0.5.0 at main tip 7d688c6 (already pushed). validate-freeze green; the v0.4.0 manifest is superseded, never mutated.

Fix-cycle headlines (derived from the frozen results, diffed against v0.4.0's)

  • Bifrost runner errors 22 → 0; kernel decisive-correct 222 → 435; smoke 115 → 117 of 118.
  • Ruby kernel decides for the first time: 0/58 → 21/58 (36 inconclusive, 1 FP). Rust gap_contract crash class (bifrost-dev#2638): 12 runner errors → 10 correct + 2 inconclusive.
  • Honest negatives, published: four new v0.10.7 false positives (bifrost-dev#2731 — C#/Go loop-carried, C#/Ruby infeasible-branch); the ten element-object crash cases resolve to inconclusive, not correct; the Java direct-propagation smoke-vs-kernel instability persists unreconciled on this build; inconclusive concentrations remain (Ruby 36, Kotlin 30, PHP/C++ 26, C#/TS 24).

First tier publications

  • Modeling (benchmark-controlled): Bifrost's Amendment-A9 category-Z cells decide 12/12 correct in their first scored run; CodeQL 24/24 ×3; Joern 14/16 (A2 partition, 4/6 categories); Semgrep 10/10 over its 5-template A3 partition.
  • Tool-native: CodeQL Java 11/12, Python 10/12, JavaScript 9/12 — misses are the preregistered hazards; Semgrep's vendored-snapshot asymmetry (Python 8/12 with 4 FPs via a shipped audit rule vs JS/Java 0/6); Bifrost and Joern decline-all with witnessed run identity (A10 rationale).
  • The landing page gains two separately fenced tier sections — the core-kernel filter that guarantees no pooling stays; declined tiers render as "declines the tier", never 0/N; a Model profile column keeps benchmark-controlled and tool-native visually distinct; vendor colors keep the fixed identity map.

Site

v0.4.0 archived with evidenceRef pinned to 86a29e0; v0.5.0 snapshot pages added; /current redirects to v0-5-0; astro check clean, 31 pages build.

Verification

cargo fmt --check · cargo test 162 · validate 852 · validate-reports 66 · validate-freeze green · generate-results --check current · astro check + build clean.

🤖 Generated with Claude Code

DavidBakerEffendi and others added 4 commits August 28, 2026 19:16
Full sequential re-run (bifrost, semgrep, joern, codeql phases) of the
852-case population: 20 Bifrost reports, 17 Semgrep, 12 Joern, 17 CodeQL.
First committed evidence for Bifrost v0.10.7 (build 44d9a5be), including
the Amendment A9 category-Z modeling cells and the A10-corrected native
rationales, with run identities witnessed from the invoked binaries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st modeling and tool-native tiers

Release notes derived entirely from results/results.json and
reports/freeze.json: Bifrost runner errors 22 -> 0, kernel decisive-correct
222 -> 435, smoke 115 -> 117 of 118, Ruby decides for the first time
(0/58 -> 21/58). Four new false positives on v0.10.7 published rather than
filtered, and the Java direct-propagation instability restated as still
unreconciled.

First publication of the modeling matrix (12 templates / 6 categories x 3
languages x 4 adapters) and the tool-native probe set (6 templates x 3
languages x 4 adapters). The site gains tier-aware sections for both, kept
in their own populations: no combined leaderboard, benchmark-controlled and
tool-native never pooled or visually conflated, model profiles labelled on
every matrix row, and declined tiers rendered as coverage rather than as
zeros. Vendor colours keep the fixed identity-keyed mapping.

Archives v0.4.0's results model pinned at 86a29e0, adds the v0.5.0 snapshot
pages, and repoints /current.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@DavidBakerEffendi
DavidBakerEffendi merged commit d6582d1 into main Aug 28, 2026
2 checks passed
@DavidBakerEffendi
DavidBakerEffendi deleted the dave/v0.5.0-freeze branch August 28, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant