Repository navigation
Version 0.2.4 - #105
Merged
Merged
Version 0.2.4#105
Conversation
tamnd
added a commit
that referenced
this pull request
Sep 6, 2026
* Fix the four jobs that have been failing on main None of these are related to the verdict work in the next commit, they are what CI has been red on since before #105 and the local gate does not cover any of them. Clippy at pedantic flags fit in the ClickBench tests for wrapping a value it always produces, so it returns the record and the call sites wrap it. bench-run called libc::geteuid behind a runtime cfg! rather than a compile time one, which compiles nowhere without geteuid, so the Windows job failed on the lib test. It is now two cfg attributed functions. libbz2-rs-sys arrives under DataFusion's default features and carries the bzip2 licence, which cargo deny rejected. It gets a scoped exception rather than a feature change, since reducing DataFusion's features to dodge a licence check would change the system under test. OpenSSF publishes no floating major tag for scorecard-action, so v2 resolves to nothing. It is pinned to the exact version with a comment saying why the major version rule cannot apply there. * Add the reproduce command and the five word verdict vocabulary Closes the first half of B3. A claim is registered as a committed TOML file under crates/bench-store/claims, and iris-bench reproduce runs the instrument that registration names and prints one of five words. The five are REPRODUCED, REPRODUCED-WITH-CAVEAT, NOT-REPRODUCED, NOT-ATTEMPTABLE and PENDING, and the set is closed. The three that follow from a measurement come out of Bar::decide, which takes the reading, the threshold and what the instrument can resolve and returns the word. Nobody picks it after seeing the number, which is the whole point of fixing the vocabulary. A caveat can turn a pass into a pass with a condition next to it and it can never turn a loss into anything softer. NOT-ATTEMPTABLE is reached two ways, by an artifact that cannot be obtained, which has to carry a citation, and by a bar narrower than the instrument can resolve. That second road is why reproduce runs the control first and grades against its interval: a harness that cannot put two copies of one buffer closer than nine percent has no business reporting a three percent difference. Run against C0002 the arithmetic lands where the prose in docs/CLAIMS.md already had it, NOT-REPRODUCED at the span iris ships and NOT-ATTEMPTABLE at a span that holds the file, which is the strongest check available that the rule matches the intent. A claim nobody registered, a reading taken before its own threshold was written down, and a claim whose instrument nobody has built are errors rather than verdicts. Everything else exits zero, because a failed reproduction is a result and not a broken build. resident gained a measure and report split so that reproduce can take a reading without printing a gate report, which also drops report from eleven arguments to two. * Fix the two remaining CI jobs The macOS test that says a control run is not refused for a span it never opens was asking gate, which also puts the result against a bar. On a shared runner two pairs can bootstrap an interval wider than any bar, so the test failed on a question it was not asking. It now asks measure, which is the part that decides whether the run is refused. DuckDB was autoloading the Parquet extension the first time a query touched a Parquet file. That puts a download inside the machine under test and lands on whatever version the server had rather than the one the lockfile names, and where the extension directory is not writable it fails outright, which is how the Windows job found it. It is linked in now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The version bump and the lockfile, and nothing else.
Everything since
v0.2.3is about whether the ClickBench numbers are comparable at all, which is the part of B2 that has to be settled before any timing from it is worth publishing. The fixture now stores the raw corpus types instead of the converted ones, so it exercises the same setups the real corpus does.clickbench answerprints the rows behind a digest, which is what turned the disagreements from a list of hashes into something that could be read. And the ten queries that do not have one answer are now named in code with the reason attached, which closed #20.Notes go on the tag.