Skip to content

Grade estimates by q-error in Flow and in the JOB gate - #275

Merged
tamnd merged 1 commit into
mainfrom
flow-q
Oct 9, 2026
Merged

tamnd merged 1 commit into
mainfrom
flow-q

Conversation

@tamnd

@tamnd tamnd commented Oct 9, 2026

Copy link
Copy Markdown
Owner

This grades the row estimates of a plan against the rows each operator produced, which is item six of J8 (tamnd/rudb#1852) and section 12.8 of the JOB notes.

In the harness, Operator now reads estimated_rows from the rudb metrics document and Flow carries how many operators had an estimate, the median q-error and the worst one, in tenths so the record stays whole numbers. The saved record appends the three fields, so older files still open and read as zero estimates. The report gets a q-error column in the "Inside the engine" table and one line with the 50th, 90th and 99th percentile and the maximum of the per query worst.

In scripts/job-gate.py there is a new qerror command. It runs each query once per engine outside any timing, with rudb writing its metrics document and DuckDB writing its JSON profile, and grades every operator both engines had an estimate for. It prints the joins, every operator, and the top join of each query. The top join matters because both engines filter a scan under a hash join by the keys the other side found, so an operator under a join can produce fewer rows than its subplan really has. Nothing filters the top join, so that one is the estimate against the true size of the whole join.

First run on server2, rule off, main at bce4a8c88, six threads:

DuckDB rudb
top join, median 141.8 19.1
top join, 90th 46,281 1,410
top join, 99th 670,390 44,335
top join, max 1,855,296 347,291
every join, median 93.0 16.0
every join, max 2,967,144 694,582

rudb's worst join estimate is closer than DuckDB's in 82 of the 113 queries. The P-error is not here yet. It needs the planner to be handed true cardinalities, which is a change in rudb.

Tested on server3 with cargo test --lib (398 passed) and clippy with -D warnings.

@tamnd tamnd added kind/feature It should do something it does not. area/joins JOB and CEB, where per-query variance is the point. area/report The per-query tables, the dashboard, the reporting rules. labels Oct 9, 2026
@tamnd
tamnd merged commit 06f5078 into main Oct 9, 2026
1 check failed
@tamnd
tamnd deleted the flow-q branch October 9, 2026 12:02
@tamnd tamnd mentioned this pull request Oct 9, 2026
10 of 11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/joins JOB and CEB, where per-query variance is the point. area/report The per-query tables, the dashboard, the reporting rules. kind/feature It should do something it does not.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant