TPC-derived benchmarks can run in two modes:
- "power" - run all the queries sequentially on the same engine, each query run has full access to the system resources
- "throughput" - run all of the queries, but randomized and over N "streams". In TPC, a stream is a concurrent client session submitting queries to the engine. Each query must share resources with the queries running on other streams.
Current behavior
- Presto (common_fixtures.py): each query runs sequentially via a single presto_cursor; iterations are repeated back-to-back on the same connection.
- Slurm path (functions.sh): one CLI container drives all queries serially.
- Reporting (post_results.py): accepts --concurrency-streams but it is not populated from benchmark output and run-sweep.sh does not pass it. Results are always submitted with concurrency_streams: 1.
Proposed behavior
- Add a
--concurrency-streams N (or equivalent) option across benchmark runners that launches N independent clients, each executing the full query set in pseudo-random order (must be deterministic)
- Records per-stream timings and aggregate throughput metrics in
benchmark_result.json.
- Propagates
concurrency_streams into the benchmark context so post_results.py can submit it without a manual CLI override.
- Add support to the Slurm NVL72 path (
launch-run.sh, run-sweep.sh) as well as local run_benchmark.sh.
TPC-derived benchmarks can run in two modes:
Current behavior
Proposed behavior
--concurrency-streamsN (or equivalent) option across benchmark runners that launches N independent clients, each executing the full query set in pseudo-random order (must be deterministic)benchmark_result.json.concurrency_streamsinto the benchmark context sopost_results.pycan submit it without a manual CLI override.launch-run.sh,run-sweep.sh) as well as localrun_benchmark.sh.