Skip to content

Add the TPC-H driver - #16

Merged
tamnd merged 5 commits into
mainfrom
tpch
Oct 7, 2026
Merged

tamnd merged 5 commits into
mainfrom
tpch

Conversation

@tamnd

@tamnd tamnd commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Part of #1.

What changes

  • src/tpch.rs is the TPC-H driver of spec/20 §20.7. The step gen runs dbgen 3.0.1 and writes the 22 queries of qgen -d to DATA/queries. Q15 uses the approved variant A of the kit, with a WITH clause in place of the view. qgen prints the row count as --LIMIT n, and the driver turns it into a LIMIT clause.
  • The step load creates the 8 tables with the types, primary keys and foreign keys of the specification. DuckDB and PostgreSQL read the same .tbl files. For a server, the driver sends the rows with COPY ... FROM STDIN through the harness client, adds the keys after the rows, and runs VACUUM ANALYZE and CHECKPOINT.
  • The step run is in src/suite.rs, which ClickBench will also use. Each query runs 3 times, and each run is a new session. The first run is cold: the driver drops the page cache, and for a server unit it also restarts the unit. The hot time is the smallest of the other runs. Each run has the counters of its cgroup. A failed query is left out of the time sums and counted as failed.
  • --answers checks the answers. An .out file of the kit at SF1 matches with the rules of TPC-H clause 2.1.3.5. A .tsv file from --save-answers of another run matches as a multiset of rows. A space at the start of a text in an .out file is kept.
  • src/pg.rs has copy_in. machines/install/tpch-tools.sh builds dbgen and qgen from the pin with makefile.suite, so a changed Makefile in the copy does not count. machines/install/postgresql.sh gives pg_checkpoint to the role bench.
  • A report name with a dot, such as tpch-sf0.1-postgresql, now works, and the driver calls dbgen and qgen with absolute paths.
  • reports/2026-10-07/68cad8fc-server3-* are the reports of the 5 smoke runs below. Their names and their first line mark them as smoke runs.
  • The README explains the command.

Test

srvtest passed on server3: clippy is clean and 50 tests pass. The tests cover the schema, the qgen output, the .out format and the rules of clause 2.1.3.5.

Smoke runs on server3, 7 October 2026. These are not baselines. server3 is a shared machine with 8 cores, and its load average was between 41 and 85 when I read it that day. PostgreSQL is REL_19_STABLE at 7d3d2db7 (19beta4), and DuckDB is 1.5.6. The script ran these commands in /root/work/tpch with RUPG_BENCH_COMMIT=68cad8fc and --smoke --report /root/work/reports --machine server3. The data of SF0.1 and SF1 was made by dbgen earlier that day.

UTC Command (shortened) Result
11:34 to 12:27 tpch --scale 0.1 --engine postgresql --unit postgresql@19-main --save-answers ans-pg-sf0.1 load 69.0 s, hot total 720.1 s, cold total 1019.0 s, 0 failed
12:27 to 12:51 instructions --set tpch --queries sf0.1/queries --engine postgresql --unit postgresql@19-main 448,092,314,188 net instructions, base 9,738,733
12:51 to 12:55 tpch --scale 0.1 --duckdb sf0.1.db --answers ans-pg-sf0.1 load 24.7 s, hot total 2.329 s, cold total 7.234 s, 22 answers checked against PostgreSQL, 0 wrong
12:55 to 12:56 instructions --set tpch --queries sf0.1/queries --duckdb sf0.1.db 2,438,500,773 net instructions, base 78,966,097
12:56 to 13:05 tpch --scale 1 --duckdb sf1.db --answers dbgen/answers --save-answers ans-duckdb-sf1 load 318.0 s, hot total 12.723 s, cold total 19.995 s, 22 answers checked against the kit, 0 wrong

The PostgreSQL hot total is mostly Q20 (465.3 s) and Q17 (236.4 s). The schema has only the keys of the specification, so these correlated subqueries scan lineitem again for each outer row. At SF1 this is not practical on server3, so the PostgreSQL run is at SF0.1. The answer of PostgreSQL at SF0.1 has no reference in the kit, so it was saved and the DuckDB run checked against it.

For the instruction counts, the median spread of the 3 runs of a query is 2.8 percent for PostgreSQL and 3.1 percent for DuckDB, and the largest spread is 51 percent (PostgreSQL Q3) and 15 percent (DuckDB Q2). The spread is (largest run minus smallest run) divided by the smallest run. On this shared machine it is above the 3 percent budget of spec/21 §21.14, so ratchet.toml stays empty.

No line of #1 is complete with this PR. The TPC-H baseline line needs the tpch machine, and the instruction line also needs the YCSB and TPC-C sets.

@tamnd tamnd added kind/feature It does not do something that it must do. area/tpch TPC-H at SF1, SF10 and SF100. labels Oct 7, 2026
@tamnd tamnd self-assigned this Oct 7, 2026
@tamnd
tamnd merged commit 5da7832 into main Oct 7, 2026
2 checks passed
@tamnd
tamnd deleted the tpch branch October 7, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/tpch TPC-H at SF1, SF10 and SF100. kind/feature It does not do something that it must do.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant