From 778313edcdee38799b94e092218ef0e4b8a0e0f4 Mon Sep 17 00:00:00 2001 From: tamnd Date: Wed, 7 Oct 2026 20:18:08 +0700 Subject: [PATCH 1/2] Add the TPC-C driver with HammerDB procedures and a statement form rupg-bench tpcc builds the schema with HammerDB 6.0, runs TPROC-C with stored procedures, and runs a driver in the harness that sends plain prepared statements. Each run measures the server cgroup over the counted minutes and checks the consistency conditions 1 to 4 of clause 3.3.2. The step check only checks them. rupg-bench fixed-set writes the YCSB and TPC-C parts of the fixed set of spec/21 section 21.14. instructions sends a server query to psql on stdin, so a long set fits, and takes --password-file. --- README.md | 14 +- machines/install/hammerdb.sh | 33 + machines/install/tpcc-roles.sh | 43 ++ src/instructions.rs | 38 +- src/main.rs | 431 +++++++++++- src/pg.rs | 19 + src/tpcc.rs | 1153 ++++++++++++++++++++++++++++++++ src/ycsb.rs | 12 + 8 files changed, 1722 insertions(+), 21 deletions(-) create mode 100755 machines/install/hammerdb.sh create mode 100755 machines/install/tpcc-roles.sh create mode 100644 src/tpcc.rs diff --git a/README.md b/README.md index b7fc6d0..3619b87 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ The procedure is [`spec/20-benchmarks.md`](https://github.com/tamnd/rupg/blob/ma ## Status -Early. The crate builds and CI is green. TPC-H, YCSB and pgbench run, and the other suites do not run yet. `rupg-bench gates` prints the twelve gates of spec/02 section 2.10, `rupg-bench pins` prints the pinned versions, `rupg-bench measure` runs the cgroup v2 runner, `rupg-bench load` runs the load cases, `rupg-bench answers` checks answer sets, `rupg-bench report` writes the report files, `rupg-bench pgbench` runs the pgbench smoke run with its check, `rupg-bench instructions` counts the instructions of each query, `rupg-bench tpch` runs TPC-H, and `rupg-bench ycsb` runs YCSB. The first milestone, M0, builds the harness and measures every baseline that a later gate uses. Every command below is the planned interface from the spec. +Early. The crate builds and CI is green. TPC-H, TPC-C, YCSB and pgbench run, and the other suites do not run yet. `rupg-bench gates` prints the twelve gates of spec/02 section 2.10, `rupg-bench pins` prints the pinned versions, `rupg-bench measure` runs the cgroup v2 runner, `rupg-bench load` runs the load cases, `rupg-bench answers` checks answer sets, `rupg-bench report` writes the report files, `rupg-bench pgbench` runs the pgbench smoke run with its check, `rupg-bench instructions` counts the instructions of each query, `rupg-bench tpch` runs TPC-H, `rupg-bench ycsb` runs YCSB, and `rupg-bench tpcc` runs TPC-C. The first milestone, M0, builds the harness and measures every baseline that a later gate uses. Every command below is the planned interface from the spec. ## The gates @@ -80,7 +80,7 @@ The commands below are the planned interface. The harness is one Rust crate with cargo run --release -- machine up 4xl cargo run --release -- clickbench --machine 4xl --rows full cargo run --release -- tpch --scale 100 --machine tpch -cargo run --release -- tpcc --form procedures --machine oltp +cargo run --release -- tpcc --unit postgresql@19-main --steps procedures,statements --sync on,off cargo run --release -- ycsb --unit postgresql@19-main --workloads a --rows 16 cargo run --release -- connections --count 1000 cargo run --release -- load --case a @@ -123,11 +123,13 @@ rupg-bench report --result idle.json --suite idle-base --machine server3 --smoke rupg-bench pgbench --unit postgresql@19-main --scale 1 --clients 4 --time 30 --smoke --report reports --machine server3 ``` -`rupg-bench instructions` is the instruction count runner of spec/21 section 21.14. It runs each query of a set under `perf stat -e instructions`. For DuckDB, perf counts the `duckdb` process with one thread. For a server, perf counts the cgroup of the server on all CPUs while `psql` sends the query, so the count is the work of the server and not the work of the client. Each query runs three times and the minimum counts. The count of `SELECT 1` in the same way is subtracted, so the net count is the work of the query. A set is a file with one query on each line, named `q0` to `q42` for ClickBench, or a directory of `.sql` files. A server needs `--engine NAME`, and the report name has the set and the engine. `--ratchet ratchet.toml` fails when one query rises by more than 3 percent or the total by more than 1 percent. `--save` writes the table for the ratchet. The ratchet is filled only on the dedicated runner, so [`ratchet.toml`](ratchet.toml) has no table yet. +`rupg-bench instructions` is the instruction count runner of spec/21 section 21.14. It runs each query of a set under `perf stat -e instructions`. For DuckDB, perf counts the `duckdb` process with one thread. For a server, perf counts the cgroup of the server on all CPUs while `psql` reads the query on its standard input and sends each statement, so the count is the work of the server and not the work of the client. `--password-file` gives `psql` a password in `PGPASSWORD`. Each query runs three times and the minimum counts. The count of `SELECT 1` in the same way is subtracted, so the net count is the work of the query. A set is a file with one query on each line, named `q0` to `q42` for ClickBench, or a directory of `.sql` files. A server needs `--engine NAME`, and the report name has the set and the engine. `--ratchet ratchet.toml` fails when one query rises by more than 3 percent or the total by more than 1 percent. `--save` writes the table for the ratchet. The ratchet is filled only on the dedicated runner, so [`ratchet.toml`](ratchet.toml) has no table yet. `rupg-bench fixed-set --out DIR` writes the YCSB and TPC-C parts of the fixed set: `DIR/ycsb/read1000.sql` has 1,000 point reads with scrambled zipfian keys, and `DIR/tpcc/neword100.sql` has 100 New-Orders in plain statements, with 5 + 2n statements each. The ClickBench part is the 43 queries on the first 1,000,000 rows of `hits`, and the TPC-H part is the 22 queries of `qgen -d` at SF 0.1. Each New-Order set adds 100 orders, so the database grows a little with each run. ```sh rupg-bench instructions --set clickbench --queries queries.sql --duckdb hits.db --duckdb-bin duckdb rupg-bench instructions --set clickbench --queries queries.sql --engine postgresql --unit postgresql@19-main --ratchet ratchet.toml +rupg-bench fixed-set --out fixed +rupg-bench instructions --set tpcc --queries fixed/tpcc --engine postgresql --unit postgresql@19-main --conn "user=tpcc dbname=tpcc" --password-file /etc/rupg-bench/tpcc.pass ``` `rupg-bench tpch` is the TPC-H driver of spec/20 section 20.7. `machines/install/tpch-tools.sh` builds `dbgen` and `qgen` 3.0.1 from the pin. The step `gen` runs `dbgen` into the data directory and writes the 22 queries of `qgen -d` to `DATA/queries`. Q15 uses the approved variant A of the kit (a `WITH` clause in place of the view). The step `load` creates the 8 tables with the types, primary keys and foreign keys of the specification and loads the same `.tbl` files into each system. For a server it adds the keys after the rows, then runs `VACUUM ANALYZE` and `CHECKPOINT`. The step `run` runs each query 3 times. The first run is cold: the driver drops the page cache, and for a server it also restarts the unit. The hot time of a query is the smallest of the other runs. Each query runs in its own cgroup, or in the cgroup of the server unit, and the result has the counters of each run. `--answers` checks the answers against the `.out` files of the kit at SF1, or against `.tsv` files from `--save-answers` of another run. @@ -143,6 +145,12 @@ rupg-bench tpch --tools tpch_tools_3.0.1/dbgen --scale 1 --data sf1 --engine pos rupg-bench ycsb --unit postgresql@19-main --records 1000000 --rows 1,16,16x64 --time 60 --sync on ``` +`rupg-bench tpcc` is the TPC-C driver of spec/20 section 20.8. It has two forms on the schema that HammerDB builds. The form `procedures` runs HammerDB TPROC-C, which calls the stored procedures that HammerDB makes in the database. The form `statements` is a driver in the harness, because HammerDB has no form with plain statements for PostgreSQL. Its terminals send prepared statements, so a New-Order makes 5 + 2n round trips: the customer and warehouse read with `BEGIN`, the district update, the insert into `orders`, the insert into `new_order`, a read of each item and an update of each stock row, and one insert of all order lines with `COMMIT`. The items are sorted, so two New-Orders do not wait on each other in a cycle, and 1 percent of New-Orders roll back on an unused item. The mix is the HammerDB mix, with no keying or think time. The step `build` drops the database `tpcc` and builds `--warehouses` with HammerDB. Each run has a ramp of `--rampup` minutes and counts `--duration` minutes, for each count of `--vu` and each value of `--sync`. NOPM is the change of `sum(d_next_o_id)` over the counted minutes, as HammerDB counts it. The server cgroup is measured over the same minutes, with the idle base first, so the result has NOPM per core, the server CPU and the device write bytes for each New-Order, and `memory.peak`. The statement form gives the p50, p95 and p99 of each transaction, and the procedure form gives the time profile of HammerDB. For a run with `synchronous_commit` on, the result prints the sync bound `0.45 x 60 x T / s`, with `s` the `fdatasync` p50 that the driver measures next to the data directory. The conditions 1 to 4 of clause 3.3.2 are checked before the runs and after each run, and a run that fails one is not a result. The step `check` only checks them. HammerDB logs in with a password, so run `machines/install/hammerdb.sh` and `machines/install/tpcc-roles.sh` first. + +```sh +rupg-bench tpcc --unit postgresql@19-main --warehouses 100 --vu 16 --rampup 5 --duration 20 --sync on,off +``` + ## The reporting rules 1. A claim has five parts: one named machine, one metric, a baseline that we measured, the same transport on both sides, and the ratio. A claim without all five is not made. diff --git a/machines/install/hammerdb.sh b/machines/install/hammerdb.sh new file mode 100755 index 0000000..7a4041a --- /dev/null +++ b/machines/install/hammerdb.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# Install the HammerDB release of pins.toml into $RUPG_BENCH_OPT/hammerdb. +# +# rupg-bench tpcc runs hammerdbcli from there for the procedure form of TPC-C (spec/20 section 20.8). HammerDB reaches PostgreSQL through libpq, so the script installs libpq5. +set -euo pipefail +# shellcheck source=machines/lib.sh +. "$(dirname "$0")/../lib.sh" + +version=$(pin hammerdb version) +url=$(pin hammerdb url) +sha=$(pin hammerdb sha256) +dir=$RUPG_BENCH_OPT/hammerdb +mkdir -p "$RUPG_BENCH_OPT/download" + +apt_install libpq5 +tarball=$RUPG_BENCH_OPT/download/$(basename "$url") +fetch "$url" "$sha" "$tarball" +if [ ! -x "$dir/hammerdbcli" ] || [ "$(cat "$dir/VERSION" 2>/dev/null)" != "$version" ]; then + rm -rf "$dir" "$dir.part" + mkdir -p "$dir.part" + tar -xzf "$tarball" -C "$dir.part" --strip-components 1 + echo "$version" >"$dir.part/VERSION" + mv "$dir.part" "$dir" +fi +[ -x "$dir/hammerdbcli" ] || die "$dir has no hammerdbcli" +# librarycheck prints whether HammerDB can load libpq. hammerdbcli waits for more input at the end of stdin, so the input ends with exit. +check=$(cd "$dir" && printf 'librarycheck\nexit\n' | timeout 120 ./hammerdbcli 2>&1 | grep -A1 'library for PostgreSQL' || true) +case $check in +*Success*) ;; +*) die "HammerDB cannot load libpq: $check" ;; +esac +log "HammerDB $version in $dir" +printf '%s\n' "$dir" diff --git a/machines/install/tpcc-roles.sh b/machines/install/tpcc-roles.sh new file mode 100755 index 0000000..9c10465 --- /dev/null +++ b/machines/install/tpcc-roles.sh @@ -0,0 +1,43 @@ +#!/bin/bash +# Make the roles that HammerDB needs in the PostgreSQL 19 cluster: hammerdb, a superuser that builds the schema, and tpcc, the owner of the schema. +# +# Usage: tpcc-roles.sh [CLUSTER]. The cluster is main by default. HammerDB makes the database tpcc and the role tpcc itself, so it needs a superuser. The roles log in on the Unix socket with a password, not with trust, because a superuser with trust would let every user of the machine in. The password is random and is kept in /etc/rupg-bench/tpcc.pass, which only root can read. rupg-bench tpcc reads it from there. +set -euo pipefail +# shellcheck source=machines/lib.sh +. "$(dirname "$0")/../lib.sh" +need_root + +major=19 +cluster=${1:-main} +passfile=/etc/rupg-bench/tpcc.pass +mkdir -p "$(dirname "$passfile")" +if [ ! -s "$passfile" ]; then + (umask 077 && head -c 24 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' >"$passfile") +fi +pass=$(cat "$passfile") +port=$(pg_lsclusters -h | awk -v v="$major" -v c="$cluster" '$1 == v && $2 == c { print $3 }') +[ -n "$port" ] || die "there is no cluster $major/$cluster" + +# The rules for the socket. They go first, so the peer rule of the package does not reject the roles. The method password asks the client for the password in clear text over the socket, so the small client of rupg-bench can use it too. +hba=/etc/postgresql/$major/$cluster/pg_hba.conf +for role in tpcc hammerdb; do + if ! grep -qE "^local[[:space:]]+all[[:space:]]+${role}[[:space:]]+password" "$hba"; then + sed -i "1i local all $role password" "$hba" + fi +done +psql_su() { + sudo -u postgres psql -h /var/run/postgresql -p "$port" -qAt "$@" +} +psql_su -c "SELECT pg_reload_conf()" >/dev/null +if [ "$(psql_su -c "SELECT count(*) FROM pg_roles WHERE rolname = 'hammerdb'")" = 0 ]; then + psql_su -c "CREATE ROLE hammerdb LOGIN SUPERUSER" +fi +# The password goes on standard input, so it is not on a command line that ps shows. It has only letters and digits, so the quotes are safe. +set_password() { + printf "ALTER ROLE %s PASSWORD '%s';\n" "$1" "$pass" | psql_su -v ON_ERROR_STOP=1 -f - +} +set_password hammerdb +if [ "$(psql_su -c "SELECT count(*) FROM pg_roles WHERE rolname = 'tpcc'")" != 0 ]; then + set_password tpcc +fi +log "roles hammerdb and tpcc in $major/$cluster on port $port, password in $passfile" diff --git a/src/instructions.rs b/src/instructions.rs index 8db55e9..1a7b59e 100644 --- a/src/instructions.rs +++ b/src/instructions.rs @@ -4,8 +4,9 @@ //! //! `ratchet.toml` holds the net count of each query from the dedicated runner. A change fails when one query rises by more than 3 percent or the total by more than 1 percent. +use std::io::Write as _; use std::path::{Path, PathBuf}; -use std::process::Command; +use std::process::{Command, Stdio}; use crate::json::Json; use crate::toml::{Doc, Value}; @@ -66,8 +67,8 @@ fn natural_key(name: &str) -> (String, u64) { pub(crate) enum Engine { /// The client program with the engine in its process: argv before the query. InProcess(Vec), - /// A server in a cgroup (the path under `/sys/fs/cgroup`) and its client: argv before the query. - Server { cgroup: String, client: Vec }, + /// A server in a cgroup (the path under `/sys/fs/cgroup`) and its client: the full argv. The client reads the query on its standard input, so a long set such as 100 New-Orders does not hit the limit of the kernel on one argument, and psql sends each statement as its own message, as a driver does. The password goes to the client in `PGPASSWORD`, so it is not on a command line. + Server { cgroup: String, client: Vec, password: Option }, } impl Engine { @@ -75,15 +76,21 @@ impl Engine { fn command(&self, sql: &str, out: &Path) -> Command { let mut c = Command::new("perf"); c.args(["stat", "-e", "instructions", "-x", ","]).arg("-o").arg(out); - let argv = match self { - Engine::InProcess(argv) => argv, - Engine::Server { cgroup, client } => { + match self { + Engine::InProcess(argv) => { + c.arg("--").args(argv).arg(sql); + c.stdin(Stdio::null()); + } + Engine::Server { cgroup, client, password } => { c.args(["-a", "-G", cgroup]); - client + if let Some(p) = password { + c.env("PGPASSWORD", p); + } + c.arg("--").args(client); + c.stdin(Stdio::piped()); } - }; - c.arg("--").args(argv).arg(sql); - c.stdout(std::process::Stdio::null()); + } + c.stdout(Stdio::null()).stderr(Stdio::piped()); c } @@ -92,7 +99,7 @@ impl Engine { Engine::InProcess(argv) => { Json::obj().with("mode", "in process").with("client", argv.clone()) } - Engine::Server { cgroup, client } => Json::obj() + Engine::Server { cgroup, client, .. } => Json::obj() .with("mode", "server cgroup") .with("cgroup", cgroup.as_str()) .with("client", client.clone()), @@ -119,7 +126,14 @@ pub(crate) fn parse_perf(text: &str) -> Result { fn count_once(engine: &Engine, sql: &str) -> Result { let out = std::env::temp_dir().join(format!("rupg-bench-perf-{}.csv", std::process::id())); let mut cmd = engine.command(sql, &out); - let result = cmd.output().map_err(|e| format!("perf: {e}")); + let result = cmd.spawn().and_then(|mut child| { + if let Some(mut stdin) = child.stdin.take() { + // psql reads the script before it writes much to stderr, so one thread is enough. A write error means that psql stopped early, and its exit status tells why. + let _ = stdin.write_all(sql.as_bytes()); + } + child.wait_with_output() + }); + let result = result.map_err(|e| format!("perf: {e}")); let text = std::fs::read_to_string(&out).unwrap_or_default(); let _ = std::fs::remove_file(&out); let result = result?; diff --git a/src/main.rs b/src/main.rs index d372084..6722e9a 100644 --- a/src/main.rs +++ b/src/main.rs @@ -24,6 +24,7 @@ mod pins; mod report; mod suite; mod toml; +mod tpcc; mod tpch; mod ycsb; @@ -58,14 +59,18 @@ commands: --report writes the report files under DIR (see report). --smoke marks the result as a smoke run, which is not a baseline. instructions --queries PATH --set NAME (--duckdb DB [--duckdb-bin B] [--threads N] | --engine NAME (--unit UNIT | --attach PATH)) - [--bin DIR] [--conn WORDS] [--repeat N] [--ratchet FILE] [--save FILE] [--smoke] [--json] [--report DIR [--machine M]] + [--bin DIR] [--conn WORDS] [--password-file F] [--repeat N] [--ratchet FILE] [--save FILE] [--smoke] [--json] [--report DIR [--machine M]] count the instructions retired for each query with perf stat (spec/21 section 21.14). PATH is a file with one query on each line or a directory of .sql files. --duckdb counts the duckdb process. --unit and --attach count the server cgroup on all - CPUs while psql from --bin sends the query. Each query runs --repeat times (3) and the + CPUs while psql from --bin reads the query on stdin and sends each statement. Each query runs --repeat times (3) and the minimum counts, minus the count of SELECT 1. --ratchet compares with [instructions.NAME] of a ratchet file and fails over the budgets (3 percent a query, 1 percent in total). - --save writes that table for this run. + --save writes that table for this run. --password-file gives psql the password in PGPASSWORD. + fixed-set --out DIR [--records N] [--warehouses N] [--seed N] + write the YCSB and TPC-C parts of the fixed set of spec/21 section 21.14 for instructions: + DIR/ycsb/read1000.sql has 1,000 YCSB point reads on a usertable of --records rows (100,000), + and DIR/tpcc/neword100.sql has 100 New-Order transactions in plain statements on --warehouses (2). tpch --tools DIR --scale S --data DIR (--duckdb DB [--duckdb-bin B] [--threads N] [--cpus LIST] | --engine NAME (--unit UNIT | --attach PATH) [--conn WORDS]) [--steps gen,load,run] [--runs N] [--no-cold] [--answers DIR] [--save-answers DIR] [--du PATH] [--smoke] [--json] [--report DIR [--machine M]] the TPC-H driver of spec/20 section 20.7. DIR of --tools is the dbgen directory that @@ -80,6 +85,18 @@ commands: workload for each row of --rows for --time seconds (60). A row is CLIENTS, or CLIENTSxDEPTH for a pipeline of DEPTH statements on each connection. --sync sets synchronous_commit on each connection. The server cgroup is measured over each run, with the idle base first. + tpcc (--unit UNIT | --attach PATH) [--hammerdb DIR] [--host H] [--port P] [--password-file F] [--steps build,procedures,statements] + [--warehouses N] [--build-vu N] [--vu 8] [--rampup MINUTES] [--duration MINUTES] [--sync on,off] [--sync-dir DIR] + [--work DIR] [--seed N] [--smoke] [--json] [--report DIR [--machine M]] + the TPC-C driver of spec/20 section 20.8. build drops the database tpcc and builds --warehouses + (10) with HammerDB from DIR (/opt/rupg-bench/hammerdb). procedures runs HammerDB TPROC-C with its + stored procedures. statements runs the driver of the harness, which sends plain statements, so a + New-Order makes 5 + 2n round trips. Each run has a ramp of --rampup minutes (5) and counts + --duration minutes (20) for each count of --vu and each value of --sync. NOPM is the change of + sum(d_next_o_id). After each run the consistency conditions 1 to 4 of clause 3.3.2 must hold. + The conditions are also checked before the runs, and --steps check only checks them. + The roles come from machines/install/tpcc-roles.sh, which writes the password file + (/etc/rupg-bench/tpcc.pass). The sync bound uses the fdatasync p50 in --sync-dir (next to the data directory). --version print the version"; fn main() -> ExitCode { @@ -127,6 +144,8 @@ fn run(argv: &[String]) -> Result<(), String> { "tpch" => tpch_command(a)?, "pgbench" => pgbench_command(a)?, "ycsb" => ycsb_command(a)?, + "tpcc" => tpcc_command(a)?, + "fixed-set" => fixed_set_command(a)?, "--version" | "-V" => println!("rupg-bench {}", env!("CARGO_PKG_VERSION")), "--help" | "-h" | "help" => println!("{USAGE}"), other => return Err(format!("usage: unknown command {other:?}\n{USAGE}")), @@ -524,9 +543,13 @@ fn instructions_command(mut a: args::Args) -> Result<(), String> { let report_dir = a.value("report"); let machine = a.value("machine"); let bin = a.value("bin").unwrap_or_else(|| "/usr/lib/postgresql/19/bin".to_owned()); - let conn = pg::Config::parse( + let mut conn = pg::Config::parse( &a.value("conn").unwrap_or_else(|| "user=bench dbname=bench".to_owned()), )?; + if let Some(f) = a.value("password-file") { + let p = std::fs::read_to_string(&f).map_err(|e| format!("{f}: {e}"))?; + conn.password = Some(p.trim().to_owned()); + } let engine_name = a.value("engine"); let engine = if let Some(db) = a.value("duckdb") { let duckdb = a.value("duckdb-bin").unwrap_or_else(|| "duckdb".to_owned()); @@ -562,8 +585,8 @@ fn instructions_command(mut a: args::Args) -> Result<(), String> { ]; client.extend(["-h".to_owned(), conn.host.clone(), "-p".to_owned(), conn.port.to_string()]); client.extend(["-U".to_owned(), conn.user.clone(), "-d".to_owned(), conn.dbname.clone()]); - client.extend(["-o".to_owned(), "/dev/null".to_owned(), "-c".to_owned()]); - instructions::Engine::Server { cgroup, client } + client.extend(["-o".to_owned(), "/dev/null".to_owned(), "-f".to_owned(), "-".to_owned()]); + instructions::Engine::Server { cgroup, client, password: conn.password.clone() } }; a.finish()?; let engine_name = match (&engine, engine_name) { @@ -1215,3 +1238,399 @@ fn ycsb_steps( Err(format!("wrong answers, the rows are not numbers: {}", failures.join("; "))) } } + +/// The `tpcc` command. +fn tpcc_command(mut a: args::Args) -> Result<(), String> { + let opt = std::env::var("RUPG_BENCH_OPT").unwrap_or_else(|_| "/opt/rupg-bench".to_owned()); + let hammerdb = PathBuf::from(a.value("hammerdb").unwrap_or_else(|| format!("{opt}/hammerdb"))); + let host = a.value("host").unwrap_or_else(|| "/var/run/postgresql".to_owned()); + let port: u16 = number(&mut a, "port", 5432)?; + let password_file = PathBuf::from( + a.value("password-file").unwrap_or_else(|| "/etc/rupg-bench/tpcc.pass".to_owned()), + ); + let steps = a.value("steps").unwrap_or_else(|| "build,procedures,statements".to_owned()); + let steps: Vec = steps.split(',').map(str::to_owned).collect(); + if let Some(bad) = + steps.iter().find(|s| !["build", "check", "procedures", "statements"].contains(&s.as_str())) + { + return Err(format!( + "usage: unknown step {bad:?}, the steps are build, check, procedures and statements" + )); + } + let warehouses: u32 = number(&mut a, "warehouses", 10)?; + let build_vu: u32 = number(&mut a, "build-vu", warehouses.clamp(1, 4))?; + let vus = a.value("vu").unwrap_or_else(|| "8".to_owned()); + let vus = vus + .split(',') + .map(|v| { + v.parse::() + .ok() + .filter(|n| *n > 0) + .ok_or(format!("usage: --vu {v:?} is not a count")) + }) + .collect::, String>>()?; + let rampup: u32 = number(&mut a, "rampup", 5)?; + let duration: u32 = number(&mut a, "duration", 20)?; + if warehouses == 0 || duration == 0 { + return Err("usage: --warehouses and --duration must be at least 1".to_owned()); + } + let syncs: Vec> = match a.value("sync") { + None => vec![None], + Some(v) => v + .split(',') + .map(|s| match s { + "on" | "off" => Ok(Some(s.to_owned())), + _ => Err(format!("usage: --sync {s:?}: give on, off or on,off")), + }) + .collect::>()?, + }; + let sync_dir = a.value("sync-dir").map(PathBuf::from); + let work = PathBuf::from( + a.value("work") + .unwrap_or_else(|| std::env::temp_dir().join("rupg-bench-tpcc").display().to_string()), + ); + let seed: u64 = number(&mut a, "seed", 1)?; + let smoke = a.flag("smoke"); + let json = a.flag("json"); + let report_dir = a.value("report"); + let machine = a.value("machine"); + if a.value_is_set("name") { + return Err("usage: a server runs in its own cgroup, so give --unit or --attach".to_owned()); + } + let cg = target(&mut a)?; + a.finish()?; + let say = |line: &str| { + if !json { + println!("{line}"); + } + }; + let meta = report::Meta::now("tpcc-postgresql", machine, None, smoke); + let mut result = meta.to_json(); + if smoke { + result = + result.with("note", "smoke run: it shows that the driver works, it is not a baseline"); + } + let hammerdb_version = std::fs::read_to_string(hammerdb.join("VERSION")) + .map(|s| s.trim().to_owned()) + .unwrap_or_default(); + result = result + .with("engine_name", "postgresql") + .with("cgroup", cg.path.display().to_string()) + .with("hammerdb_version", hammerdb_version) + .with("rampup_min", rampup) + .with("duration_min", duration) + .with("seed", seed) + .with("mix", "HammerDB: New-Order 10/23, Payment 10/23, Delivery, Stock-Level and Order-Status 1/23 each, no keying or think time") + .with( + "kernel", + std::fs::read_to_string("/proc/sys/kernel/osrelease").unwrap_or_default().trim(), + ); + let plan = TpccPlan { + hammerdb, + work, + steps, + warehouses, + build_vu, + vus, + rampup, + duration, + syncs, + sync_dir, + seed, + }; + let outcome = tpcc::Server::new(host, port, password_file) + .and_then(|server| tpcc_steps(&cg, &server, &plan, &mut result, &say)); + if let Err(e) = &outcome { + result = result.with("error", e.as_str()); + } + if let Some(dir) = &report_dir { + let (md, js) = report::write(Path::new(dir), &result)?; + say(&format!("report {}\n {}", md.display(), js.display())); + } + if json { + print!("{}", result.pretty()); + } + let removed = cg.remove(); + outcome.and(removed) +} + +/// The settings of the `tpcc` command. +#[derive(Debug)] +struct TpccPlan { + hammerdb: PathBuf, + work: PathBuf, + steps: Vec, + warehouses: u32, + build_vu: u32, + vus: Vec, + rampup: u32, + duration: u32, + syncs: Vec>, + sync_dir: Option, + seed: u64, +} + +/// The steps of the `tpcc` command. They add their numbers to `result`. +fn tpcc_steps( + cg: &Cgroup, + server: &tpcc::Server, + plan: &TpccPlan, + result: &mut Json, + say: &dyn Fn(&str), +) -> Result<(), String> { + let add = |result: &mut Json, key: &str, value: Json| { + *result = std::mem::replace(result, Json::Null).with(key, value); + }; + let has = |s: &str| plan.steps.iter().any(|x| x == s); + let mut admin = pg::Conn::connect(&server.admin())?; + add(result, "server_version", admin.server_version.as_str().into()); + let setting = |db: &mut pg::Conn, name: &str| -> Result { + let r = db.query(&format!("SHOW {name}"))?; + Ok(r.rows.first().and_then(|r| r.first()).cloned().flatten().unwrap_or_default()) + }; + let server_sync = setting(&mut admin, "synchronous_commit")?; + let data_dir = setting(&mut admin, "data_directory")?; + say(&format!("server {}, synchronous_commit {server_sync}", admin.server_version)); + if has("build") { + say(&format!( + "build {} warehouses with {} virtual users", + plan.warehouses, plan.build_vu + )); + admin.simple("DROP DATABASE IF EXISTS tpcc WITH (FORCE)")?; + admin.simple("DROP ROLE IF EXISTS tpcc")?; + let script = tpcc::build_script(server, plan.warehouses, plan.build_vu)?; + let interval = cgroup::Interval::start(cg)?; + let started = std::time::Instant::now(); + let out = tpcc::run_hammerdb(&plan.hammerdb, &plan.work, "build", &script, server, cg, say); + let secs = started.elapsed().as_secs_f64(); + let usage = interval.finish()?; + let out = out?; + if !out.lines.iter().any(|l| l.contains("TPCC SCHEMA COMPLETE")) { + return Err(format!("the HammerDB build did not complete, see {}", out.log.display())); + } + let size = admin.query("SELECT pg_database_size('tpcc')")?; + let size: Option = size + .rows + .first() + .and_then(|r| r.first().cloned().flatten()) + .and_then(|v| v.parse().ok()); + say(&format!(" done in {secs:.1} s")); + add( + result, + "build", + Json::obj() + .with("warehouses", plan.warehouses) + .with("virtual_users", plan.build_vu) + .with("secs", (secs * 10.0).round() / 10.0) + .with("database_bytes", size) + .with("server", usage.to_json()), + ); + } + let mut db = pg::Conn::connect(&server.tpcc())?; + let r = db.query("SELECT count(*) FROM warehouse")?; + let warehouses: u32 = r + .rows + .first() + .and_then(|r| r.first().cloned().flatten()) + .and_then(|v| v.parse().ok()) + .filter(|n| *n > 0) + .ok_or("the database tpcc has no warehouses. Run the build step")?; + add(result, "warehouses", warehouses.into()); + let mut failures = Vec::new(); + let conditions = tpcc::consistency(&mut db)?; + let bad: u64 = conditions.iter().map(|c| c.failed).sum(); + say(&format!("check before the runs: conditions 1 to 4, {bad} failures")); + if bad > 0 { + failures.push(format!("before the runs: {bad} units fail the consistency conditions")); + } + add(result, "check_before", tpcc::conditions_json(&conditions)); + let forms: Vec<&str> = ["procedures", "statements"].into_iter().filter(|f| has(f)).collect(); + if forms.is_empty() { + return if failures.is_empty() { Ok(()) } else { Err(failures.join("; ")) }; + } + let effective = |s: &Option| s.clone().unwrap_or_else(|| server_sync.clone()); + let fsync = if plan.syncs.iter().any(|s| effective(s) != "off") { + let dir = plan.sync_dir.clone().unwrap_or_else(|| { + Path::new(&data_dir).parent().map_or(PathBuf::from(&data_dir), Path::to_path_buf) + }); + let p50 = tpcc::fdatasync_p50(&dir, 200)?; + say(&format!( + "fsync fdatasync p50 {:.1} us in {}", + p50.as_secs_f64() * 1e6, + dir.display() + )); + add( + result, + "fdatasync", + Json::obj() + .with("dir", dir.display().to_string()) + .with("rounds", 200u64) + .with("write_bytes", 8192u64) + .with("p50_us", (p50.as_secs_f64() * 1e7).round() / 10.0), + ); + Some(p50) + } else { + None + }; + let base = cgroup::idle_base(cg, cgroup::IDLE_BASE)?; + add(result, "idle_base", base.to_json()); + let cores = tpcc::server_cores(cg); + add(result, "server_cores", cores.into()); + let mut rows = Vec::new(); + for sync in &plan.syncs { + let used = effective(sync); + for &vu in &plan.vus { + for form in &forms { + say(&format!("run {form}, {vu} virtual users, synchronous_commit {used}")); + let mut row = Json::obj() + .with("form", *form) + .with("virtual_users", vu) + .with("synchronous_commit", used.as_str()) + .with("synchronous_commit_set_by_driver", sync.is_some()); + let (nopm, tpm, usage, new_orders); + if *form == "procedures" { + if let Some(v) = sync { + admin.simple(&format!("ALTER ROLE tpcc SET synchronous_commit = {v}"))?; + } + let script = tpcc::run_script(server, vu, plan.rampup, plan.duration)?; + let name = format!("run-{form}-{vu}-{used}"); + let out = tpcc::run_hammerdb( + &plan.hammerdb, + &plan.work, + &name, + &script, + server, + cg, + say, + ); + if sync.is_some() { + admin.simple("ALTER ROLE tpcc RESET synchronous_commit")?; + } + let out = out?; + let (n, t) = tpcc::parse_result(&out.lines).ok_or(format!( + "HammerDB printed no TEST RESULT, see {}", + out.log.display() + ))?; + let u = out.measured.ok_or(format!( + "HammerDB printed no start and end of the timed interval, see {}", + out.log.display() + ))?; + let profile = std::fs::read_to_string(out.work.join("hdbxtprofile.log")) + .map(|t| tpcc::parse_profile(&t)) + .unwrap_or_default(); + let mut times = Json::obj(); + for (name, j) in profile { + times = times.with(&name, j); + } + row = row + .with( + "tpm_source", + "HammerDB TEST RESULT: pg_stat_database commits and rollbacks", + ) + .with("time_profile", times) + .with( + "time_profile_note", + "HammerDB times each procedure call over the whole run, the ramp too", + ); + new_orders = n as f64 * f64::from(plan.duration); + (nopm, tpm, usage) = (n as f64, t as f64, u); + } else { + let s = tpcc::RunSettings { + warehouses, + terminals: vu, + rampup: Duration::from_secs(u64::from(plan.rampup) * 60), + duration: Duration::from_secs(u64::from(plan.duration) * 60), + seed: plan.seed, + sync: sync.clone(), + }; + let run = tpcc::run_statements(&server.tpcc(), &s, cg)?; + let minutes = run.measured.as_secs_f64() / 60.0; + let mut times = Json::obj(); + for (tx, name) in tpcc::TXS { + times = times.with(name, run.tally.latency[tx as usize].to_json()); + } + row = row + .with( + "tpm_source", + "the transactions that the terminals finished in the interval", + ) + .with("latency", times) + .with("new_order_rollbacks", run.tally.rolled_back) + .with("deliveries_skipped_districts", run.tally.deliveries_skipped) + .with("retries", run.tally.retries) + .with("errors", run.tally.errors) + .with("first_error", run.tally.first_error.clone()); + if run.tally.errors > 0 { + failures.push(format!( + "statements with {vu} virtual users: {} errors, the first: {}", + run.tally.errors, + run.tally.first_error.clone().unwrap_or_default() + )); + } + new_orders = run.new_orders as f64; + nopm = run.new_orders as f64 / minutes; + tpm = run.tally.transactions() as f64 / minutes; + usage = run.usage; + } + let per_core = nopm / f64::from(cores); + let cpu_per_no = (new_orders > 0.0).then(|| usage.cpu_usec as f64 / new_orders); + let w_per_no = + usage.wbytes.filter(|_| new_orders > 0.0).map(|b| b as f64 / new_orders); + let bound = fsync.filter(|_| used != "off").and_then(|p| tpcc::sync_bound(vu, p)); + let conditions = tpcc::consistency(&mut db)?; + let bad: u64 = conditions.iter().map(|c| c.failed).sum(); + row = row + .with("nopm", nopm.round()) + .with("tpm", tpm.round()) + .with("nopm_per_core", per_core.round()) + .with("server_cpu_usec_per_new_order", cpu_per_no.map(|v| v.round())) + .with("server_write_bytes_per_new_order", w_per_no.map(|v| v.round())) + .with("server_memory_peak", usage.memory_peak) + .with("sync_bound_nopm", bound.map(f64::round)) + .with("server", usage.to_json()) + .with("check", tpcc::conditions_json(&conditions)); + say(&format!( + " {nopm:.0} NOPM ({per_core:.0} a core), {tpm:.0} TPM, cpu {} us a New-Order, sync bound {}", + cpu_per_no.map_or("-".to_owned(), |v| format!("{v:.0}")), + bound.map_or("-".to_owned(), |v| format!("{v:.0} NOPM")) + )); + say(&format!(" check: conditions 1 to 4, {bad} failures")); + if bad > 0 { + failures.push(format!( + "{form} with {vu} virtual users, synchronous_commit {used}: {bad} units fail the consistency conditions" + )); + } + rows.push(row); + } + } + } + add(result, "rows", rows.into()); + if failures.is_empty() { + Ok(()) + } else { + Err(format!("the run is not a result: {}", failures.join("; "))) + } +} + +/// The `fixed-set` command. +fn fixed_set_command(mut a: args::Args) -> Result<(), String> { + let out = PathBuf::from(a.value("out").ok_or("usage: fixed-set needs --out DIR")?); + let records: u64 = number(&mut a, "records", 100_000)?; + let warehouses: u32 = number(&mut a, "warehouses", 2)?; + let seed: u64 = number(&mut a, "seed", 1)?; + a.finish()?; + if records == 0 || warehouses == 0 { + return Err("usage: --records and --warehouses must be at least 1".to_owned()); + } + let files = [ + (out.join("ycsb"), "read1000.sql", ycsb::fixed_reads(records, 1000, seed)), + (out.join("tpcc"), "neword100.sql", tpcc::fixed_new_orders(warehouses, 100, seed)), + ]; + for (dir, name, text) in files { + std::fs::create_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?; + let path = dir.join(name); + std::fs::write(&path, text).map_err(|e| format!("{}: {e}", path.display()))?; + println!("{}", path.display()); + } + Ok(()) +} diff --git a/src/pg.rs b/src/pg.rs index 6ee23ec..274de98 100644 --- a/src/pg.rs +++ b/src/pg.rs @@ -411,6 +411,25 @@ impl Conn { } } + /// As `next_done`, but it keeps the values of the rows as text. The column names are empty, because `queue` does not ask for a description. + pub(crate) fn next_rows(&mut self) -> Result, PgError> { + let mut rows = Rows::default(); + let mut err = None; + loop { + let (tag, msg) = self.read_message()?; + match tag { + b'2' | b'n' | b'N' | b'S' => {} + b'D' => rows.rows.push(parse_data_row(&msg)?), + b'C' => rows.tag = cstring_at(&msg, 0)?.0, + b'E' => err = Some(parse_error(&msg)), + b'Z' => return Ok(err.map_or(Ok(rows), Err)), + other => { + return Err(client_error(format!("unexpected message {:?}", other as char))); + } + } + } + } + fn send(&mut self, tag: u8, body: &[u8]) -> Result<(), PgError> { let len = u32::try_from(body.len() + 4).map_err(|_| client_error("a message is too long"))?; diff --git a/src/tpcc.rs b/src/tpcc.rs new file mode 100644 index 0000000..214e4ce --- /dev/null +++ b/src/tpcc.rs @@ -0,0 +1,1153 @@ +//! The TPC-C driver of spec/20 section 20.8. +//! +//! It has two forms on the same schema. The procedure form runs HammerDB TPROC-C, which calls the stored procedures that HammerDB makes in the database. The statement form is a driver in this module that sends plain statements from the client, so a New-Order makes 5 + 2n round trips (document 02 section 2.7.2). HammerDB has no statement form for PostgreSQL, so the harness has its own. Both forms run on the schema that HammerDB builds, and both are measured in the same way: NOPM from the change of `sum(d_next_o_id)`, as HammerDB counts it, the server cgroup over the measured interval after the ramp, and the consistency conditions 1 to 4 of the TPC-C specification, clause 3.3.2, after each run. + +use std::fmt::Write as _; +use std::io::{BufRead, BufReader, Write as _}; +use std::os::unix::fs::FileExt; +use std::path::{Path, PathBuf}; +use std::process::{Command, Stdio}; +use std::sync::{Barrier, OnceLock}; +use std::time::{Duration, Instant}; + +use crate::cgroup::{Cgroup, Interval, Usage}; +use crate::json::Json; +use crate::pg::{Config, Conn, PgError, Rows}; +use crate::ycsb::{Histogram, Rng}; + +/// The five transactions, in the order of the reports. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Tx { + NewOrder, + Payment, + OrderStatus, + Delivery, + StockLevel, +} + +/// The transactions with the names that HammerDB gives its procedures. +pub(crate) const TXS: [(Tx, &str); 5] = [ + (Tx::NewOrder, "neword"), + (Tx::Payment, "payment"), + (Tx::OrderStatus, "ostat"), + (Tx::Delivery, "delivery"), + (Tx::StockLevel, "slev"), +]; + +/// The mix of the HammerDB driver: a number from 1 to 23, New-Order for 1 to 10, Payment for 11 to 20, then one each for Delivery, Stock-Level and Order-Status. Each of the last three is above the 4 percent minimum of clause 5.2.3. +fn pick(rng: &mut Rng) -> Tx { + match uniform(rng, 1, 23) { + 1..=10 => Tx::NewOrder, + 11..=20 => Tx::Payment, + 21 => Tx::Delivery, + 22 => Tx::StockLevel, + _ => Tx::OrderStatus, + } +} + +/// A number in [lo, hi]. +fn uniform(rng: &mut Rng, lo: u32, hi: u32) -> u32 { + lo + rng.below(u64::from(hi - lo + 1)) as u32 +} + +/// NURand(A, x, y) of clause 2.1.6. +fn nurand(rng: &mut Rng, a: u32, x: u32, y: u32, c: u32) -> u32 { + (((uniform(rng, 0, a) | uniform(rng, x, y)) + c) % (y - x + 1)) + x +} + +const SYLLABLES: [&str; 10] = + ["BAR", "OUGHT", "ABLE", "PRI", "PRES", "ESE", "ANTI", "CALLY", "ATION", "EING"]; + +/// The customer last name of clause 4.3.2.3 for a number from 0 to 999. +pub(crate) fn last_name(n: u32) -> String { + let mut s = String::new(); + for d in [n / 100, n / 10 % 10, n % 10] { + s.push_str(SYLLABLES[d as usize]); + } + s +} + +/// The constants C of NURand for one run (clause 2.1.6). +#[derive(Clone, Copy, Debug)] +struct Consts { + c_last: u32, + c_id: u32, + ol_i_id: u32, +} + +/// Cents from a numeric value as text, for example `12.5` or `-3.07`. +pub(crate) fn cents(s: &str) -> Option { + let (neg, s) = s.strip_prefix('-').map_or((false, s), |r| (true, r)); + let (int, frac) = s.split_once('.').unwrap_or((s, "")); + let mut frac = frac.to_owned(); + if frac.len() > 2 || !frac.chars().all(|c| c.is_ascii_digit()) { + return None; + } + while frac.len() < 2 { + frac.push('0'); + } + let v = int.parse::().ok()? * 100 + frac.parse::().ok()?; + Some(if neg { -v } else { v }) +} + +/// Cents as a numeric value, for example `1250` as `12.50`. +pub(crate) fn money(c: i64) -> String { + let sign = if c < 0 { "-" } else { "" }; + format!("{sign}{}.{:02}", c.abs() / 100, c.abs() % 100) +} + +/// A PostgreSQL array of text values. +pub(crate) fn text_array<'a>(items: impl IntoIterator) -> String { + let mut s = String::from("{"); + for (i, v) in items.into_iter().enumerate() { + if i > 0 { + s.push(','); + } + s.push('"'); + for c in v.chars() { + if c == '"' || c == '\\' { + s.push('\\'); + } + s.push(c); + } + s.push('"'); + } + s.push('}'); + s +} + +/// The sync bound of document 03 section 3.12.2: `0.45 x 60 x T / s` New-Orders a minute, with `T` terminals and `s` the `fdatasync` p50 in seconds. +pub(crate) fn sync_bound(terminals: u32, fdatasync: Duration) -> Option { + let s = fdatasync.as_secs_f64(); + (s > 0.0).then(|| 0.45 * 60.0 * f64::from(terminals) / s) +} + +/// The p50 of `rounds` writes of 8 KiB at the start of a file in `dir`, each followed by `fdatasync`. The file is removed at the end. +pub(crate) fn fdatasync_p50(dir: &Path, rounds: u32) -> Result { + let path = dir.join(format!(".rupg-bench-fdatasync-{}", std::process::id())); + let file = std::fs::OpenOptions::new() + .create(true) + .truncate(true) + .read(true) + .write(true) + .open(&path) + .map_err(|e| format!("{}: {e}", path.display()))?; + let block = [0x5au8; 8192]; + let mut times = Vec::with_capacity(rounds as usize); + let mut failed = None; + for _ in 0..rounds { + let t = Instant::now(); + if let Err(e) = file.write_all_at(&block, 0).and_then(|()| file.sync_data()) { + failed = Some(e); + break; + } + times.push(t.elapsed()); + } + drop(file); + let removed = std::fs::remove_file(&path); + if let Some(e) = failed { + return Err(format!("{}: {e}", path.display())); + } + removed.map_err(|e| format!("{}: {e}", path.display()))?; + times.sort_unstable(); + times.get(times.len() / 2).copied().ok_or_else(|| "no rounds".to_owned()) +} + +/// The number of CPUs in a list such as `0-3,6`. +pub(crate) fn cpu_count(list: &str) -> Option { + let mut n = 0; + for part in list.trim().split(',').filter(|p| !p.is_empty()) { + let (a, b) = part.split_once('-').unwrap_or((part, part)); + let (a, b): (u32, u32) = (a.parse().ok()?, b.parse().ok()?); + n += b.checked_sub(a)? + 1; + } + (n > 0).then_some(n) +} + +/// The CPUs that the server can use: `cpuset.cpus.effective` of its cgroup, or all CPUs of the machine. +pub(crate) fn server_cores(cg: &Cgroup) -> u32 { + std::fs::read_to_string(cg.path.join("cpuset.cpus.effective")) + .ok() + .and_then(|s| cpu_count(&s)) + .or_else(|| std::thread::available_parallelism().ok().map(|n| n.get() as u32)) + .unwrap_or(1) +} + +// The procedure form. + +/// Where the server is and how HammerDB logs in. HammerDB makes the database `tpcc` and the role `tpcc` with the superuser `hammerdb`. Both roles have the password in `password_file` (see machines/install/tpcc-roles.sh). +#[derive(Clone, Debug)] +pub(crate) struct Server { + pub(crate) host: String, + pub(crate) port: u16, + pub(crate) password_file: PathBuf, + pub(crate) password: String, +} + +impl Server { + pub(crate) fn new(host: String, port: u16, password_file: PathBuf) -> Result { + let password = std::fs::read_to_string(&password_file) + .map_err(|e| { + format!("{}: {e}. Run machines/install/tpcc-roles.sh", password_file.display()) + })? + .trim() + .to_owned(); + if password.is_empty() { + return Err(format!("{} is empty", password_file.display())); + } + Ok(Server { host, port, password_file, password }) + } + + /// The connection of the role `tpcc` to the database `tpcc`. + pub(crate) fn tpcc(&self) -> Config { + self.config("tpcc", "tpcc") + } + + /// The connection of the superuser `hammerdb` to the database `postgres`. + pub(crate) fn admin(&self) -> Config { + self.config("hammerdb", "postgres") + } + + fn config(&self, user: &str, dbname: &str) -> Config { + Config { + host: self.host.clone(), + port: self.port, + user: user.to_owned(), + dbname: dbname.to_owned(), + password: Some(self.password.clone()), + } + } + + /// The `dbset` and `diset` lines that every script has. The script reads the password from the file, so the password is not in the script. + fn tcl_head(&self) -> Result { + Ok(format!( + "set pass [string trim [read [open {}]]]\ndbset db pg\ndbset bm TPC-C\ndiset connection pg_host {}\ndiset connection pg_port {}\ndiset connection pg_sslmode disable\ndiset tpcc pg_superuser hammerdb\ndiset tpcc pg_superuserpass $pass\ndiset tpcc pg_defaultdbase postgres\ndiset tpcc pg_user tpcc\ndiset tpcc pg_pass $pass\ndiset tpcc pg_dbase tpcc\ndiset tpcc pg_storedprocs true\n", + tcl_word(&self.password_file.display().to_string())?, + tcl_word(&self.host)?, + self.port + )) + } +} + +/// A word for a Tcl script, in braces. Words with braces, backslashes or line breaks are refused. +pub(crate) fn tcl_word(s: &str) -> Result { + if s.is_empty() || s.contains(['{', '}', '\\', '\n', '\r']) { + return Err(format!("{s:?} cannot be a word of a HammerDB script")); + } + Ok(format!("{{{s}}}")) +} + +/// The script that builds the schema with `vu` virtual users. HammerDB partitions `order_line` from 200 warehouses, as in its sample script. +pub(crate) fn build_script(s: &Server, warehouses: u32, vu: u32) -> Result { + let mut t = s.tcl_head()?; + let _ = write!( + t, + "diset tpcc pg_count_ware {warehouses}\ndiset tpcc pg_num_vu {vu}\ndiset tpcc pg_tspace pg_default\ndiset tpcc pg_partition {}\nbuildschema\nputs \"RUPG-BENCH BUILD DONE\"\n", + warehouses >= 200 + ); + Ok(t) +} + +/// The script of a timed run: a ramp of `rampup` minutes, then `duration` minutes that count, with the time profile on. It does not vacuum at the end. +pub(crate) fn run_script( + s: &Server, + vu: u32, + rampup: u32, + duration: u32, +) -> Result { + let mut t = s.tcl_head()?; + let _ = write!( + t, + "diset tpcc pg_driver timed\ndiset tpcc pg_total_iterations 10000000\ndiset tpcc pg_rampup {rampup}\ndiset tpcc pg_duration {duration}\ndiset tpcc pg_allwarehouse false\ndiset tpcc pg_timeprofile true\ndiset tpcc pg_vacuum false\nloadscript\nvuset vu {vu}\nvuset logtotemp 0\nvucreate\nvurun\nvudestroy\nputs \"RUPG-BENCH RUN DONE\"\n" + ); + Ok(t) +} + +/// The output of one HammerDB script. +#[derive(Debug)] +pub(crate) struct HammerOut { + pub(crate) lines: Vec, + /// The server cgroup from "Rampup complete" to "Test complete", for a run. + pub(crate) measured: Option, + pub(crate) log: PathBuf, + pub(crate) work: PathBuf, +} + +/// Runs `hammerdbcli auto` on `script` in a new directory `work/name`, which is also its TMP, so the settings of an earlier run do not carry over. The log is `work/name/hammerdb.log`, with the password replaced. For a run, the server cgroup is measured between the lines of the monitor that start and end the timed interval. +pub(crate) fn run_hammerdb( + hammerdb: &Path, + work: &Path, + name: &str, + script: &str, + server: &Server, + cg: &Cgroup, + say: &dyn Fn(&str), +) -> Result { + let dir = work.join(name); + if dir.exists() { + std::fs::remove_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?; + } + std::fs::create_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?; + let tcl = dir.join("script.tcl"); + std::fs::write(&tcl, script).map_err(|e| format!("{}: {e}", tcl.display()))?; + let log_path = dir.join("hammerdb.log"); + let mut log = + std::fs::File::create(&log_path).map_err(|e| format!("{}: {e}", log_path.display()))?; + let stderr = log.try_clone().map_err(|e| e.to_string())?; + let mut child = Command::new(hammerdb.join("hammerdbcli")) + .arg("auto") + .arg(&tcl) + .current_dir(hammerdb) + .env("TMP", &dir) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::from(stderr)) + .spawn() + .map_err(|e| format!("{}/hammerdbcli: {e}", hammerdb.display()))?; + let out = child.stdout.take().ok_or("no stdout")?; + let mut lines = Vec::new(); + let mut interval = None; + let mut measured = None; + let mut failed = None; + for line in BufReader::new(out).lines() { + let line = match line { + Ok(l) => l.replace(&server.password, "********"), + Err(e) => { + failed = Some(e.to_string()); + break; + } + }; + let _ = writeln!(log, "{line}"); + if line.contains("Rampup complete, Taking start Transaction Count") { + say(" rampup complete, the measured interval starts"); + interval = Some(Interval::start(cg)?); + } else if line.contains("Test complete, Taking end Transaction Count") + && let Some(i) = interval.take() + { + measured = Some(i.finish()?); + } + if line.contains("Error") || line.contains("FINISHED FAILED") { + say(&format!(" hammerdb: {line}")); + } + lines.push(line); + } + let status = child.wait().map_err(|e| e.to_string())?; + if let Some(e) = failed { + return Err(format!("reading the output of hammerdbcli: {e}")); + } + if !status.success() { + return Err(format!("hammerdbcli failed: {status}, see {}", log_path.display())); + } + Ok(HammerOut { lines, measured, log: log_path, work: dir }) +} + +/// NOPM and TPM from "TEST RESULT : System achieved N NOPM from M PostgreSQL TPM". +pub(crate) fn parse_result(lines: &[String]) -> Option<(u64, u64)> { + let line = lines.iter().find(|l| l.contains("TEST RESULT"))?; + let words: Vec<&str> = line.split_whitespace().collect(); + let at = |w: &str| words.iter().position(|x| *x == w); + let nopm = words.get(at("NOPM")?.checked_sub(1)?)?.parse().ok()?; + let tpm = words.get(at("TPM")?.checked_sub(2)?)?.parse().ok()?; + Some((nopm, tpm)) +} + +/// The summary of the HammerDB time profile (`hdbxtprofile.log`): for each procedure, its calls and its times in milliseconds. The profile covers the whole run of each virtual user, the ramp too. +pub(crate) fn parse_profile(text: &str) -> Vec<(String, Json)> { + let mut out = Vec::new(); + let Some(start) = text.rfind(">>>>> SUMMARY OF") else { + return out; + }; + let mut cur: Option<(String, Json)> = None; + for line in text[start..].lines().skip(1) { + if line.starts_with("+-") { + break; + } + if let Some(name) = line.strip_prefix(">>>>> PROC: ") { + out.extend(cur.take()); + cur = Some((name.trim().to_ascii_lowercase(), Json::obj())); + continue; + } + let Some((_, j)) = cur.as_mut() else { continue }; + for part in line.split('\t') { + let Some((k, v)) = part.split_once(':') else { continue }; + let (key, v) = (k.trim().to_ascii_lowercase(), v.trim()); + let value = if key == "calls" { + v.parse::().ok().map(|n| ("calls".to_owned(), Json::from(n))) + } else if let Some(ms) = v.strip_suffix("ms").and_then(|n| n.parse::().ok()) { + Some((format!("{key}_ms"), Json::from(ms))) + } else { + v.strip_suffix('%') + .and_then(|n| n.parse::().ok()) + .map(|pc| (format!("{key}_percent"), Json::from(pc))) + }; + if let Some((name, value)) = value { + *j = std::mem::replace(j, Json::Null).with(&name, value); + } + } + } + out.extend(cur); + out +} + +// The consistency conditions. + +/// One consistency condition of clause 3.3.2: how many warehouses or districts it checked and how many fail. +#[derive(Clone, Debug, PartialEq, Eq)] +pub(crate) struct Condition { + pub(crate) number: u32, + pub(crate) text: &'static str, + pub(crate) checked: u64, + pub(crate) failed: u64, +} + +/// The SQL of conditions 1 to 4. Each returns the units it checked and the units that fail. Condition 2 compares `max(NO_O_ID)` only in a district that still has rows in NEW-ORDER, because Delivery can empty a district. +const CONDITIONS: [(u32, &str, &str); 4] = [ + ( + 1, + "W_YTD = sum(D_YTD) for each warehouse", + "SELECT count(*), count(*) FILTER (WHERE w.w_ytd <> d.s) FROM warehouse w JOIN (SELECT d_w_id, sum(d_ytd) AS s FROM district GROUP BY d_w_id) d ON d.d_w_id = w.w_id", + ), + ( + 2, + "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "SELECT count(*), count(*) FILTER (WHERE o.m IS DISTINCT FROM d.d_next_o_id - 1 OR (n.m IS NOT NULL AND n.m <> d.d_next_o_id - 1)) FROM district d LEFT JOIN (SELECT o_w_id, o_d_id, max(o_id) AS m FROM orders GROUP BY 1, 2) o ON o.o_w_id = d.d_w_id AND o.o_d_id = d.d_id LEFT JOIN (SELECT no_w_id, no_d_id, max(no_o_id) AS m FROM new_order GROUP BY 1, 2) n ON n.no_w_id = d.d_w_id AND n.no_d_id = d.d_id", + ), + ( + 3, + "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "SELECT count(*), count(*) FILTER (WHERE hi - lo + 1 <> n) FROM (SELECT max(no_o_id) AS hi, min(no_o_id) AS lo, count(*) AS n FROM new_order GROUP BY no_w_id, no_d_id) x", + ), + ( + 4, + "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "SELECT count(*), count(*) FILTER (WHERE o.s IS DISTINCT FROM l.c) FROM (SELECT o_w_id AS w, o_d_id AS d, sum(o_ol_cnt) AS s FROM orders GROUP BY 1, 2) o FULL JOIN (SELECT ol_w_id AS w, ol_d_id AS d, count(*) AS c FROM order_line GROUP BY 1, 2) l ON l.w = o.w AND l.d = o.d", + ), +]; + +/// Runs conditions 1 to 4 of clause 3.3.2. +pub(crate) fn consistency(db: &mut Conn) -> Result, String> { + let mut out = Vec::new(); + for (number, text, sql) in CONDITIONS { + let r = db.query(sql)?; + let get = |i: usize| -> Result { + r.rows + .first() + .and_then(|row| row.get(i).cloned().flatten()) + .and_then(|v| v.parse().ok()) + .ok_or_else(|| format!("condition {number}: no count")) + }; + out.push(Condition { number, text, checked: get(0)?, failed: get(1)? }); + } + Ok(out) +} + +pub(crate) fn conditions_json(c: &[Condition]) -> Json { + c.iter() + .map(|c| { + Json::obj() + .with("condition", c.number) + .with("text", c.text) + .with("checked", c.checked) + .with("failed", c.failed) + }) + .collect::>() + .into() +} + +/// The sum of `d_next_o_id`. Its change over an interval is the count of New-Orders that committed in it. +pub(crate) fn next_order_sum(db: &mut Conn) -> Result { + let r = db.query("SELECT sum(d_next_o_id) FROM district")?; + r.rows + .first() + .and_then(|row| row.first().cloned().flatten()) + .and_then(|v| v.parse().ok()) + .ok_or_else(|| "district is empty".to_owned()) +} + +// The statement form. + +/// The statements of the statement form. Each connection prepares them once. +const STATEMENTS: [(&str, &str); 24] = [ + ("begin", "BEGIN"), + ("commit", "COMMIT"), + ( + "no_wc", + "SELECT c_discount, c_last, c_credit, w_tax FROM customer, warehouse WHERE w_id = $1::int AND c_w_id = $1::int AND c_d_id = $2::int AND c_id = $3::int", + ), + ( + "no_d", + "UPDATE district SET d_next_o_id = d_next_o_id + 1 WHERE d_w_id = $1::int AND d_id = $2::int RETURNING d_next_o_id - 1, d_tax", + ), + ( + "no_o", + "INSERT INTO orders (o_id, o_d_id, o_w_id, o_c_id, o_entry_d, o_ol_cnt, o_all_local) VALUES ($1::int, $2::int, $3::int, $4::int, current_timestamp, $5::int, $6::int)", + ), + ( + "no_no", + "INSERT INTO new_order (no_o_id, no_d_id, no_w_id) VALUES ($1::int, $2::int, $3::int)", + ), + ("no_i", "SELECT i_price, i_name, i_data FROM item WHERE i_id = $1::int"), + ( + "no_s", + "UPDATE stock SET s_quantity = CASE WHEN s_quantity >= $3::int + 10 THEN s_quantity - $3::int ELSE s_quantity - $3::int + 91 END, s_ytd = s_ytd + $3::int, s_order_cnt = s_order_cnt + 1, s_remote_cnt = s_remote_cnt + $4::int WHERE s_i_id = $1::int AND s_w_id = $2::int RETURNING s_quantity, (ARRAY[s_dist_01, s_dist_02, s_dist_03, s_dist_04, s_dist_05, s_dist_06, s_dist_07, s_dist_08, s_dist_09, s_dist_10])[$5::int], s_data", + ), + ( + "no_ol", + "INSERT INTO order_line (ol_o_id, ol_d_id, ol_w_id, ol_number, ol_i_id, ol_supply_w_id, ol_quantity, ol_amount, ol_dist_info) SELECT $1::int, $2::int, $3::int, n, i, s, q, a, d FROM unnest($4::int[], $5::int[], $6::int[], $7::int[], $8::numeric[], $9::text[]) AS t(n, i, s, q, a, d)", + ), + ( + "pay_w", + "UPDATE warehouse SET w_ytd = w_ytd + $2::numeric WHERE w_id = $1::int RETURNING w_name, w_street_1, w_street_2, w_city, w_state, w_zip", + ), + ( + "pay_d", + "UPDATE district SET d_ytd = d_ytd + $3::numeric WHERE d_w_id = $1::int AND d_id = $2::int RETURNING d_name, d_street_1, d_street_2, d_city, d_state, d_zip", + ), + ( + "pay_cn", + "SELECT c_id FROM customer WHERE c_w_id = $1::int AND c_d_id = $2::int AND c_last = $3::text ORDER BY c_first", + ), + ( + "pay_c", + "UPDATE customer SET c_balance = c_balance - $4::numeric, c_ytd_payment = c_ytd_payment + $4::numeric, c_payment_cnt = c_payment_cnt + 1, c_data = CASE WHEN c_credit = 'BC' THEN left($5::text || ' ' || c_data, 500) ELSE c_data END WHERE c_w_id = $1::int AND c_d_id = $2::int AND c_id = $3::int RETURNING c_first, c_middle, c_last, c_street_1, c_street_2, c_city, c_state, c_zip, c_phone, c_since, c_credit, c_credit_lim, c_discount, c_balance, left(c_data, 200)", + ), + ( + "pay_h", + "INSERT INTO history (h_c_id, h_c_d_id, h_c_w_id, h_d_id, h_w_id, h_date, h_amount, h_data) VALUES ($1::int, $2::int, $3::int, $4::int, $5::int, current_timestamp, $6::numeric, $7::text)", + ), + ( + "os_cn", + "SELECT c_id, c_balance, c_first, c_middle, c_last FROM customer WHERE c_w_id = $1::int AND c_d_id = $2::int AND c_last = $3::text ORDER BY c_first", + ), + ( + "os_c", + "SELECT c_id, c_balance, c_first, c_middle, c_last FROM customer WHERE c_w_id = $1::int AND c_d_id = $2::int AND c_id = $3::int", + ), + ( + "os_o", + "SELECT o_id, o_carrier_id, o_entry_d, o_ol_cnt FROM orders WHERE o_w_id = $1::int AND o_d_id = $2::int AND o_c_id = $3::int ORDER BY o_id DESC LIMIT 1", + ), + ( + "os_ol", + "SELECT ol_i_id, ol_supply_w_id, ol_quantity, ol_amount, ol_delivery_d FROM order_line WHERE ol_w_id = $1::int AND ol_d_id = $2::int AND ol_o_id = $3::int", + ), + ( + "dl_no", + "DELETE FROM new_order WHERE no_w_id = $1::int AND no_d_id = $2::int AND no_o_id = (SELECT min(no_o_id) FROM new_order WHERE no_w_id = $1::int AND no_d_id = $2::int) RETURNING no_o_id", + ), + ( + "dl_o", + "UPDATE orders SET o_carrier_id = $4::int WHERE o_w_id = $1::int AND o_d_id = $2::int AND o_id = $3::int RETURNING o_c_id", + ), + ( + "dl_ol", + "WITH u AS (UPDATE order_line SET ol_delivery_d = current_timestamp WHERE ol_w_id = $1::int AND ol_d_id = $2::int AND ol_o_id = $3::int RETURNING ol_amount) SELECT coalesce(sum(ol_amount), 0), count(*) FROM u", + ), + ( + "dl_c", + "UPDATE customer SET c_balance = c_balance + $4::numeric, c_delivery_cnt = c_delivery_cnt + 1 WHERE c_w_id = $1::int AND c_d_id = $2::int AND c_id = $3::int", + ), + ("sl_d", "SELECT d_next_o_id FROM district WHERE d_w_id = $1::int AND d_id = $2::int"), + ( + "sl_c", + "SELECT count(DISTINCT s_i_id) FROM order_line, stock WHERE ol_w_id = $1::int AND ol_d_id = $2::int AND ol_o_id < $3::int AND ol_o_id >= $3::int - 20 AND s_w_id = $1::int AND s_i_id = ol_i_id AND s_quantity < $4::int", + ), +]; + +/// An error for a result that breaks a rule of the transaction, such as a customer that is not there. It is not a server error, but it counts as an error of the run. +fn wrong(message: String) -> PgError { + PgError { code: "wrong".to_owned(), message } +} + +/// One round trip: the statements go to the server together, and the result of each comes back. The first error of the server is returned after all results are read, so the connection stays in step. An error with an empty code is a failure of the connection. +fn trip(c: &mut Conn, list: &[(&str, Vec)]) -> Result, PgError> { + for (name, params) in list { + let p: Vec<&[u8]> = params.iter().map(|s| s.as_bytes()).collect(); + c.queue(name, &p)?; + } + c.flush()?; + let mut out = Vec::with_capacity(list.len()); + let mut first = None; + for _ in list { + match c.next_rows()? { + Ok(r) => out.push(r), + Err(e) => { + first = first.or(Some(e)); + out.push(Rows::default()); + } + } + } + first.map_or(Ok(out), Err) +} + +/// The value at `row` and `col`, or an error that names the statement. +fn val<'a>(r: &'a Rows, row: usize, col: usize, what: &str) -> Result<&'a str, PgError> { + r.rows + .get(row) + .and_then(|x| x.get(col)) + .and_then(|v| v.as_deref()) + .ok_or_else(|| wrong(format!("{what}: no value at row {row}, column {col}"))) +} + +fn num(s: &str, what: &str) -> Result { + s.parse().map_err(|_| wrong(format!("{what}: {s:?} is not a number"))) +} + +fn args(v: &[&dyn std::fmt::Display]) -> Vec { + v.iter().map(|x| x.to_string()).collect() +} + +/// What one terminal knows. +struct Terminal<'a> { + c: &'a mut Conn, + rng: Rng, + k: Consts, + w: u32, + warehouses: u32, +} + +/// What a transaction did, for the counts. +enum Done { + Committed, + /// A New-Order with an unused item, which rolls back (clause 2.4.2.3). + RolledBack, + /// A Delivery that found no new order in some districts. + Skipped(u32), +} + +impl Terminal<'_> { + /// A warehouse other than the home warehouse, or the home warehouse when there is one. + fn other_warehouse(&mut self) -> u32 { + if self.warehouses == 1 { + return self.w; + } + let o = uniform(&mut self.rng, 1, self.warehouses - 1); + if o >= self.w { o + 1 } else { o } + } + + fn new_order(&mut self) -> Result { + let (w, rng) = (self.w, &mut self.rng); + let d = uniform(rng, 1, 10); + let c_id = nurand(rng, 1023, 1, 3000, self.k.c_id); + let n = uniform(rng, 5, 15); + let rollback = uniform(rng, 1, 100) == 1; + let mut items = Vec::with_capacity(n as usize); + for j in 0..n { + let rng = &mut self.rng; + let i = if rollback && j == n - 1 { + 100_001 + } else { + nurand(rng, 8191, 1, 100_000, self.k.ol_i_id) + }; + let remote = uniform(rng, 1, 100) == 1; + let qty = uniform(rng, 1, 10); + let supply = if remote { self.other_warehouse() } else { w }; + items.push((i, supply, qty)); + } + // In the order of the keys, so two New-Orders do not wait on each other in a cycle. The unused item is the largest, so it is still the last. + items.sort_unstable(); + let all_local = u32::from(items.iter().all(|x| x.1 == w)); + let r = trip(self.c, &[("begin", vec![]), ("no_wc", args(&[&w, &d, &c_id]))])?; + val(&r[1], 0, 0, "New-Order customer")?; + let r = trip(self.c, &[("no_d", args(&[&w, &d]))])?; + let o_id: u32 = num(val(&r[0], 0, 0, "New-Order district")?, "d_next_o_id")?; + trip(self.c, &[("no_o", args(&[&o_id, &d, &w, &c_id, &n, &all_local]))])?; + trip(self.c, &[("no_no", args(&[&o_id, &d, &w]))])?; + let mut amounts = Vec::with_capacity(items.len()); + let mut dists = Vec::with_capacity(items.len()); + for &(i, supply, qty) in &items { + let r = trip(self.c, &[("no_i", args(&[&i]))])?; + if r[0].rows.is_empty() { + // The unused item: the transaction rolls back. + self.c.simple("ROLLBACK")?; + if i != 100_001 { + return Err(wrong(format!("New-Order: item {i} is not there"))); + } + return Ok(Done::RolledBack); + } + let price = cents(val(&r[0], 0, 0, "item")?) + .ok_or_else(|| wrong("New-Order: the item price is not a number".to_owned()))?; + let remote = u32::from(supply != w); + let r = trip(self.c, &[("no_s", args(&[&i, &supply, &qty, &remote, &d]))])?; + dists.push(val(&r[0], 0, 1, "stock")?.to_owned()); + amounts.push(money(price * i64::from(qty))); + } + let list = |f: &dyn Fn(usize) -> String| { + format!("{{{}}}", (0..items.len()).map(f).collect::>().join(",")) + }; + let numbers = list(&|j| (j + 1).to_string()); + let ids = list(&|j| items[j].0.to_string()); + let supplies = list(&|j| items[j].1.to_string()); + let qtys = list(&|j| items[j].2.to_string()); + let amounts = format!("{{{}}}", amounts.join(",")); + let dists = text_array(dists.iter().map(String::as_str)); + let mut p = args(&[&o_id, &d, &w]); + p.extend([numbers, ids, supplies, qtys, amounts, dists]); + let r = trip(self.c, &[("no_ol", p), ("commit", vec![])])?; + if r[0].tag != format!("INSERT 0 {}", items.len()) { + return Err(wrong(format!("New-Order: order_line insert gave {:?}", r[0].tag))); + } + Ok(Done::Committed) + } + + fn payment(&mut self) -> Result { + let (w, rng) = (self.w, &mut self.rng); + let d = uniform(rng, 1, 10); + let home = uniform(rng, 1, 100) <= 85; + let by_name = uniform(rng, 1, 100) <= 60; + let amount = money(i64::from(uniform(rng, 100, 500_000))); + let name = last_name(nurand(rng, 255, 0, 999, self.k.c_last)); + let id = nurand(rng, 1023, 1, 3000, self.k.c_id); + let (cw, cd) = if home || self.warehouses == 1 { + (w, d) + } else { + let cd = uniform(&mut self.rng, 1, 10); + (self.other_warehouse(), cd) + }; + let r = trip(self.c, &[("begin", vec![]), ("pay_w", args(&[&w, &amount]))])?; + let w_name = val(&r[1], 0, 0, "Payment warehouse")?.to_owned(); + let r = trip(self.c, &[("pay_d", args(&[&w, &d, &amount]))])?; + let d_name = val(&r[0], 0, 0, "Payment district")?.to_owned(); + let c_id: u32 = if by_name { + let r = trip(self.c, &[("pay_cn", args(&[&cw, &cd, &name]))])?; + let n = r[0].rows.len(); + if n == 0 { + return Err(wrong(format!("Payment: no customer {name} in {cw}/{cd}"))); + } + num(val(&r[0], (n - 1) / 2, 0, "Payment customer")?, "c_id")? + } else { + id + }; + let note = format!("{c_id} {cd} {cw} {d} {w} {amount}"); + let r = trip(self.c, &[("pay_c", args(&[&cw, &cd, &c_id, &amount, ¬e]))])?; + val(&r[0], 0, 0, "Payment customer update")?; + let h_data = format!("{w_name} {d_name}"); + let r = trip( + self.c, + &[("pay_h", args(&[&c_id, &cd, &cw, &d, &w, &amount, &h_data])), ("commit", vec![])], + )?; + if r[0].tag != "INSERT 0 1" { + return Err(wrong(format!("Payment: history insert gave {:?}", r[0].tag))); + } + Ok(Done::Committed) + } + + fn order_status(&mut self) -> Result { + let (w, rng) = (self.w, &mut self.rng); + let d = uniform(rng, 1, 10); + let by_name = uniform(rng, 1, 100) <= 60; + let first = if by_name { + let name = last_name(nurand(rng, 255, 0, 999, self.k.c_last)); + ("os_cn", args(&[&w, &d, &name])) + } else { + let id = nurand(rng, 1023, 1, 3000, self.k.c_id); + ("os_c", args(&[&w, &d, &id])) + }; + let r = trip(self.c, &[("begin", vec![]), first])?; + let n = r[1].rows.len(); + if n == 0 { + return Err(wrong(format!("Order-Status: no customer in {w}/{d}"))); + } + let c_id: u32 = num(val(&r[1], (n - 1) / 2, 0, "Order-Status customer")?, "c_id")?; + let r = trip(self.c, &[("os_o", args(&[&w, &d, &c_id]))])?; + if r[0].rows.is_empty() { + trip(self.c, &[("commit", vec![])])?; + return Ok(Done::Committed); + } + let o_id: u32 = num(val(&r[0], 0, 0, "Order-Status order")?, "o_id")?; + let lines: usize = num(val(&r[0], 0, 3, "Order-Status order")?, "o_ol_cnt")?; + let r = trip(self.c, &[("os_ol", args(&[&w, &d, &o_id])), ("commit", vec![])])?; + if r[0].rows.len() != lines { + return Err(wrong(format!( + "Order-Status: order {w}/{d}/{o_id} has o_ol_cnt {lines} and {} order lines", + r[0].rows.len() + ))); + } + Ok(Done::Committed) + } + + fn delivery(&mut self) -> Result { + let w = self.w; + let carrier = uniform(&mut self.rng, 1, 10); + let mut skipped = 0; + trip(self.c, &[("begin", vec![])])?; + for d in 1..=10u32 { + let r = trip(self.c, &[("dl_no", args(&[&w, &d]))])?; + let Some(o) = r[0].rows.first().and_then(|row| row.first().cloned().flatten()) else { + skipped += 1; + continue; + }; + let o_id: u32 = num(&o, "no_o_id")?; + let r = trip( + self.c, + &[("dl_o", args(&[&w, &d, &o_id, &carrier])), ("dl_ol", args(&[&w, &d, &o_id]))], + )?; + let c_id: u32 = num(val(&r[0], 0, 0, "Delivery order")?, "o_c_id")?; + let total = val(&r[1], 0, 0, "Delivery order lines")?.to_owned(); + if val(&r[1], 0, 1, "Delivery order lines")? == "0" { + return Err(wrong(format!("Delivery: order {w}/{d}/{o_id} has no order lines"))); + } + let r = trip(self.c, &[("dl_c", args(&[&w, &d, &c_id, &total]))])?; + if r[0].tag != "UPDATE 1" { + return Err(wrong(format!("Delivery: customer {w}/{d}/{c_id}: {:?}", r[0].tag))); + } + } + trip(self.c, &[("commit", vec![])])?; + Ok(if skipped > 0 { Done::Skipped(skipped) } else { Done::Committed }) + } + + fn stock_level(&mut self) -> Result { + let w = self.w; + let d = uniform(&mut self.rng, 1, 10); + let threshold = uniform(&mut self.rng, 10, 20); + let r = trip(self.c, &[("begin", vec![]), ("sl_d", args(&[&w, &d]))])?; + let next: u32 = num(val(&r[1], 0, 0, "Stock-Level district")?, "d_next_o_id")?; + let r = trip(self.c, &[("sl_c", args(&[&w, &d, &next, &threshold])), ("commit", vec![])])?; + val(&r[0], 0, 0, "Stock-Level count")?; + Ok(Done::Committed) + } +} + +/// The settings of one run of the statement form. +#[derive(Clone, Debug)] +pub(crate) struct RunSettings { + pub(crate) warehouses: u32, + pub(crate) terminals: u32, + pub(crate) rampup: Duration, + pub(crate) duration: Duration, + pub(crate) seed: u64, + pub(crate) sync: Option, +} + +/// The numbers of the terminals over the measured interval. +#[derive(Clone, Debug, Default)] +pub(crate) struct Tally { + pub(crate) latency: [Histogram; 5], + pub(crate) rolled_back: u64, + pub(crate) deliveries_skipped: u64, + pub(crate) retries: u64, + pub(crate) errors: u64, + pub(crate) first_error: Option, +} + +impl Tally { + fn merge(&mut self, o: Tally) { + for (a, b) in self.latency.iter_mut().zip(&o.latency) { + a.merge(b); + } + self.rolled_back += o.rolled_back; + self.deliveries_skipped += o.deliveries_skipped; + self.retries += o.retries; + self.errors += o.errors; + self.first_error = self.first_error.take().or(o.first_error); + } + + pub(crate) fn transactions(&self) -> u64 { + self.latency.iter().map(Histogram::count).sum() + } +} + +/// The result of one run of the statement form. +#[derive(Debug)] +pub(crate) struct StatementRun { + pub(crate) tally: Tally, + pub(crate) usage: Usage, + /// The change of `sum(d_next_o_id)` over the measured interval. + pub(crate) new_orders: u64, + pub(crate) measured: Duration, +} + +/// Runs the statement form: `terminals` connections, each on its home warehouse, with no keying or think time, as HammerDB runs. The server cgroup and `sum(d_next_o_id)` are read at the end of the ramp and at the end of the interval. A transaction counts when it starts and ends in the interval. +pub(crate) fn run_statements( + config: &Config, + s: &RunSettings, + cg: &Cgroup, +) -> Result { + let mut seed_rng = Rng::new(s.seed); + let k = Consts { + c_last: uniform(&mut seed_rng, 0, 255), + c_id: uniform(&mut seed_rng, 0, 1023), + ol_i_id: uniform(&mut seed_rng, 0, 8191), + }; + let n = s.terminals as usize; + let ready = Barrier::new(n + 1); + let go = Barrier::new(n + 1); + let window: OnceLock<(Instant, Instant)> = OnceLock::new(); + let mut db = Conn::connect(config)?; + std::thread::scope(|scope| { + let handles: Vec<_> = (0..n) + .map(|i| { + let (ready, go, window) = (&ready, &go, &window); + scope.spawn(move || terminal(config, s, k, i as u32, (ready, go), window)) + }) + .collect(); + ready.wait(); + let now = Instant::now(); + let (t0, t1) = (now + s.rampup, now + s.rampup + s.duration); + let _ = window.set((t0, t1)); + go.wait(); + std::thread::sleep(t0.saturating_duration_since(Instant::now())); + let measured = (|| -> Result<(Interval, u64, Instant), String> { + let before = next_order_sum(&mut db)?; + let start = Instant::now(); + Ok((Interval::start(cg)?, before, start)) + })(); + std::thread::sleep(t1.saturating_duration_since(Instant::now())); + let after = measured.and_then(|(interval, before, start)| { + let usage = interval.finish()?; + let elapsed = start.elapsed(); + Ok((usage, next_order_sum(&mut db)?.saturating_sub(before), elapsed)) + }); + let mut total = Tally::default(); + let mut failed = None; + for h in handles { + match h.join() { + Ok(Ok(t)) => total.merge(t), + Ok(Err(e)) => failed = failed.or(Some(e)), + Err(_) => failed = failed.or(Some("a terminal thread panicked".to_owned())), + } + } + if let Some(e) = failed { + return Err(e); + } + let (usage, new_orders, measured) = after?; + Ok(StatementRun { tally: total, usage, new_orders, measured }) + }) +} + +fn terminal( + config: &Config, + s: &RunSettings, + k: Consts, + i: u32, + (ready, go): (&Barrier, &Barrier), + window: &OnceLock<(Instant, Instant)>, +) -> Result { + let setup = (|| -> Result { + let mut c = Conn::connect(config)?; + if let Some(v) = &s.sync { + c.simple(&format!("SET synchronous_commit = {v}"))?; + } + for (name, sql) in STATEMENTS { + c.prepare(name, sql)?; + } + Ok(c) + })(); + ready.wait(); + go.wait(); + let mut c = setup.map_err(|e| format!("terminal {i}: {e}"))?; + let &(t0, t1) = window.get().ok_or("no window")?; + let mut t = Terminal { + c: &mut c, + rng: Rng::new(s.seed.wrapping_mul(0x9E37_79B9).wrapping_add(u64::from(i) + 1)), + k, + w: i % s.warehouses + 1, + warehouses: s.warehouses, + }; + let mut tally = Tally::default(); + loop { + let start = Instant::now(); + if start >= t1 { + break; + } + let tx = pick(&mut t.rng); + let mut retries = 0; + let outcome = loop { + let r = match tx { + Tx::NewOrder => t.new_order(), + Tx::Payment => t.payment(), + Tx::OrderStatus => t.order_status(), + Tx::Delivery => t.delivery(), + Tx::StockLevel => t.stock_level(), + }; + match r { + Ok(done) => break Ok(done), + Err(e) if e.code.is_empty() => return Err(format!("terminal {i}: {e}")), + Err(e) => { + t.c.simple("ROLLBACK").map_err(|e| format!("terminal {i}: {e}"))?; + if (e.code == "40001" || e.code == "40P01") && retries < 100 { + retries += 1; + continue; + } + break Err(e); + } + } + }; + let end = Instant::now(); + if start < t0 || end > t1 { + continue; + } + tally.retries += retries; + match outcome { + Ok(done) => { + tally.latency[tx as usize].record(end.duration_since(start).as_nanos() as u64); + match done { + Done::Committed => {} + Done::RolledBack => tally.rolled_back += 1, + Done::Skipped(n) => tally.deliveries_skipped += u64::from(n), + } + } + Err(e) => { + tally.errors += 1; + if tally.first_error.is_none() { + tally.first_error = Some(e.to_string()); + } + } + } + } + Ok(tally) +} + +/// `count` New-Order transactions in plain statements, with 5 + 2n statements each, as the terminals of the statement form send them. The statements that need the new order number read it from `district`, so the text needs no client between the statements. No order rolls back. They are the TPC-C part of the fixed set of spec/21 section 21.14. +pub(crate) fn fixed_new_orders(warehouses: u32, count: u32, seed: u64) -> String { + let mut rng = Rng::new(seed); + let c_id_c = uniform(&mut rng, 0, 1023); + let item_c = uniform(&mut rng, 0, 8191); + let mut s = String::new(); + for k in 0..count { + let w = k % warehouses + 1; + let d = uniform(&mut rng, 1, 10); + let c = nurand(&mut rng, 1023, 1, 3000, c_id_c); + let n = uniform(&mut rng, 5, 15); + let mut items: Vec<(u32, u32, u32)> = (0..n) + .map(|_| { + let i = nurand(&mut rng, 8191, 1, 100_000, item_c); + let remote = warehouses > 1 && uniform(&mut rng, 1, 100) == 1; + let supply = if remote { w % warehouses + 1 } else { w }; + (i, supply, uniform(&mut rng, 1, 10)) + }) + .collect(); + items.sort_unstable(); + let local = u32::from(items.iter().all(|x| x.1 == w)); + let district = format!("FROM district WHERE d_w_id = {w} AND d_id = {d}"); + let _ = writeln!(s, "BEGIN;"); + let _ = writeln!( + s, + "SELECT c_discount, c_last, c_credit, w_tax FROM customer, warehouse WHERE w_id = {w} AND c_w_id = {w} AND c_d_id = {d} AND c_id = {c};" + ); + let _ = writeln!( + s, + "UPDATE district SET d_next_o_id = d_next_o_id + 1 WHERE d_w_id = {w} AND d_id = {d} RETURNING d_next_o_id - 1, d_tax;" + ); + let _ = writeln!( + s, + "INSERT INTO orders (o_id, o_d_id, o_w_id, o_c_id, o_entry_d, o_ol_cnt, o_all_local) SELECT d_next_o_id - 1, {d}, {w}, {c}, current_timestamp, {n}, {local} {district};" + ); + let _ = writeln!( + s, + "INSERT INTO new_order (no_o_id, no_d_id, no_w_id) SELECT d_next_o_id - 1, {d}, {w} {district};" + ); + for &(i, supply, q) in &items { + let _ = writeln!(s, "SELECT i_price, i_name, i_data FROM item WHERE i_id = {i};"); + let _ = writeln!( + s, + "UPDATE stock SET s_quantity = CASE WHEN s_quantity >= {q} + 10 THEN s_quantity - {q} ELSE s_quantity - {q} + 91 END, s_ytd = s_ytd + {q}, s_order_cnt = s_order_cnt + 1, s_remote_cnt = s_remote_cnt + {} WHERE s_i_id = {i} AND s_w_id = {supply} RETURNING s_quantity, s_dist_{d:02}, s_data;", + u32::from(supply != w) + ); + } + let col = |f: &dyn Fn(usize) -> u32| { + (0..items.len()).map(|j| f(j).to_string()).collect::>().join(",") + }; + let _ = writeln!( + s, + "INSERT INTO order_line (ol_o_id, ol_d_id, ol_w_id, ol_number, ol_i_id, ol_supply_w_id, ol_quantity, ol_amount, ol_dist_info) SELECT d.d_next_o_id - 1, {d}, {w}, t.n, t.i, t.s, t.q, t.q * it.i_price, st.s_dist_{d:02} FROM district d, unnest(ARRAY[{}], ARRAY[{}], ARRAY[{}], ARRAY[{}]) AS t(n, i, s, q) JOIN item it ON it.i_id = t.i JOIN stock st ON st.s_i_id = t.i AND st.s_w_id = t.s WHERE d.d_w_id = {w} AND d.d_id = {d};", + col(&|j| j as u32 + 1), + col(&|j| items[j].0), + col(&|j| items[j].1), + col(&|j| items[j].2) + ); + let _ = writeln!(s, "COMMIT;"); + } + s +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn names_and_nurand() { + assert_eq!(last_name(0), "BARBARBAR"); + assert_eq!(last_name(371), "PRICALLYOUGHT"); + assert_eq!(last_name(999), "EINGEINGEING"); + let mut rng = Rng::new(7); + for _ in 0..10_000 { + let v = nurand(&mut rng, 1023, 1, 3000, 259); + assert!((1..=3000).contains(&v)); + let v = nurand(&mut rng, 255, 0, 999, 0); + assert!(v <= 999); + } + let mut counts = [0u32; 5]; + for _ in 0..23_000 { + counts[pick(&mut rng) as usize] += 1; + } + // About 10000, 10000, 1000, 1000 and 1000. + assert!(counts[0] > 9_500 && counts[0] < 10_500, "{counts:?}"); + assert!(counts[2] > 800 && counts[2] < 1_200, "{counts:?}"); + } + + #[test] + fn money_and_arrays() { + assert_eq!(cents("12.34"), Some(1234)); + assert_eq!(cents("12.3"), Some(1230)); + assert_eq!(cents("-0.07"), Some(-7)); + assert_eq!(cents("7"), Some(700)); + assert_eq!(cents("1.234"), None); + assert_eq!(money(1234), "12.34"); + assert_eq!(money(-7), "-0.07"); + assert_eq!(money(500_000), "5000.00"); + assert_eq!(text_array(["ab", "c\"d", "e\\f"]), r#"{"ab","c\"d","e\\f"}"#); + assert_eq!(cpu_count("0-7\n"), Some(8)); + assert_eq!(cpu_count("0-3,6"), Some(5)); + assert_eq!(cpu_count(""), None); + let b = sync_bound(16, Duration::from_millis(2)).unwrap_or_default(); + assert!((b - 216_000.0).abs() < 1e-6); + } + + #[test] + fn hammerdb_scripts_and_output() { + let s = Server { + host: "/var/run/postgresql".to_owned(), + port: 5433, + password_file: PathBuf::from("/etc/rupg-bench/tpcc.pass"), + password: "secret".to_owned(), + }; + let b = build_script(&s, 4, 2).unwrap_or_default(); + assert!(b.contains("diset tpcc pg_count_ware 4\n")); + assert!(b.contains("diset connection pg_port 5433\n")); + assert!(b.contains("[open {/etc/rupg-bench/tpcc.pass}]")); + assert!(!b.contains("secret")); + let r = run_script(&s, 8, 5, 20).unwrap_or_default(); + assert!(r.contains("diset tpcc pg_rampup 5\ndiset tpcc pg_duration 20\n")); + assert!(r.contains("vuset vu 8\n")); + assert!(tcl_word("a{b").is_err()); + let lines = vec![ + "Vuser 1:2 Active Virtual Users configured".to_owned(), + "Vuser 1:TEST RESULT : System achieved 1785 NOPM from 4109 PostgreSQL TPM".to_owned(), + ]; + assert_eq!(parse_result(&lines), Some((1785, 4109))); + assert_eq!(parse_result(&lines[..1]), None); + let profile = ">>>>> VIRTUAL USER 2 : ELAPSED TIME : 120522ms\n>>>>> PROC: NEWORD\nCALLS: 9\tMIN: 1.0ms\tAVG: 2.0ms\tMAX: 3.0ms\tTOTAL: 18.0ms\nP99: 3.0ms\tP95: 3.0ms\tP75: 2.5ms\tP50: 2.0ms\tP25: 1.5ms\tSD: 0.5ms\tRATIO: 50.0%\n+-+-+\n>>>>> SUMMARY OF 2 ACTIVE VIRTUAL USERS : MEDIAN ELAPSED TIME : 127146ms\n>>>>> PROC: NEWORD\nCALLS: 3427\tMIN: 1.730ms\tAVG: 36.332ms\tMAX: 1220.165ms\tTOTAL: 124508.229ms\nP99: 210.118ms\tP95: 104.921ms\tP75: 50.942ms\tP50: 19.126ms\tP25: 9.919ms\tSD: 49.663ms\tRATIO: 48.963%\n>>>>> PROC: OSTAT\nCALLS: 332\tMIN: 0.485ms\tAVG: 15.077ms\tMAX: 225.815ms\tTOTAL: 5005.542ms\nP99: 112.663ms\tP95: 70.415ms\tP75: 13.620ms\tP50: 4.711ms\tP25: 1.395ms\tSD: 26.570ms\tRATIO: 1.968%\n+-+-+-+\n"; + let p = parse_profile(profile); + assert_eq!(p.len(), 2); + assert_eq!(p[0].0, "neword"); + assert_eq!(p[0].1.get("calls"), Some(&Json::from(3427u64))); + assert_eq!(p[0].1.get("p50_ms"), Some(&Json::from(19.126))); + assert_eq!(p[1].0, "ostat"); + assert_eq!(p[1].1.get("ratio_percent"), Some(&Json::from(1.968))); + } + + #[test] + fn fixed_new_orders_have_5_plus_2n_statements() { + let text = fixed_new_orders(2, 100, 1); + let orders: Vec<&str> = text.split("BEGIN;\n").skip(1).collect(); + assert_eq!(orders.len(), 100); + for o in orders { + let items = o.matches("FROM item WHERE").count(); + assert!((5..=15).contains(&items)); + // BEGIN and COMMIT are not round trips of their own: they go with the first and the last statement. + let statements = o.matches(";\n").count() - 1; + assert_eq!(statements, 5 + 2 * items, "{o}"); + } + } +} diff --git a/src/ycsb.rs b/src/ycsb.rs index bb5254b..fb9650e 100644 --- a/src/ycsb.rs +++ b/src/ycsb.rs @@ -629,6 +629,18 @@ pub(crate) fn run(config: &Config, s: &RunSettings, cg: &Cgroup) -> Result<(Tall }) } +/// `count` point reads of the keys that the scrambled zipfian distribution picks from `records` rows, one statement on each line. They are the YCSB part of the fixed set of spec/21 section 21.14. +pub(crate) fn fixed_reads(records: u64, count: u32, seed: u64) -> String { + let keys = ScrambledZipfian::new(records); + let mut rng = Rng::new(seed); + let mut s = String::new(); + for _ in 0..count { + let k = key_name(keys.next(&mut rng)); + s.push_str(&format!("SELECT * FROM usertable WHERE ycsb_key = '{k}';\n")); + } + s +} + /// The result of the check after a run. #[derive(Clone, Debug, Default, PartialEq, Eq)] pub(crate) struct FinalCheck { From efc9746217856815bd635214b48a33aa691b86b3 Mon Sep 17 00:00:00 2001 From: tamnd Date: Wed, 7 Oct 2026 21:00:12 +0700 Subject: [PATCH 2/2] Add the reports of the TPC-C and fixed set smoke runs on server3 --- ...r3-instructions-tpcc-postgresql-smoke.json | 48 ++ ...ver3-instructions-tpcc-postgresql-smoke.md | 40 ++ ...r3-instructions-ycsb-postgresql-smoke.json | 48 ++ ...ver3-instructions-ycsb-postgresql-smoke.md | 40 ++ ...41f7e63-server3-tpcc-postgresql-smoke.json | 553 ++++++++++++++++++ .../b41f7e63-server3-tpcc-postgresql-smoke.md | 89 +++ 6 files changed, 818 insertions(+) create mode 100644 reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.json create mode 100644 reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.md create mode 100644 reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.json create mode 100644 reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.md create mode 100644 reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.json create mode 100644 reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.md diff --git a/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.json b/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.json new file mode 100644 index 0000000..99ac167 --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.json @@ -0,0 +1,48 @@ +{ + "suite": "instructions-tpcc-postgresql", + "machine": "server3", + "date": "2026-10-07", + "commit": "b41f7e63", + "smoke": true, + "note": "smoke run: it shows that the runner works, it is not the ratchet", + "set": "tpcc", + "engine_name": "postgresql", + "queries_from": "/root/work/fixed2/tpcc", + "repeat": 3, + "kernel": "6.8.0-106-generic", + "engine": { + "mode": "server cgroup", + "cgroup": "system.slice/system-postgresql.slice/postgresql@19-ycsb.service", + "client": [ + "/usr/lib/postgresql/19/bin/psql", + "-X", + "-q", + "-v", + "ON_ERROR_STOP=1", + "-h", + "/var/run/postgresql", + "-p", + "5433", + "-U", + "tpcc", + "-d", + "tpcc", + "-o", + "/dev/null", + "-f", + "-" + ] + }, + "counts": { + "base_runs": [52594897, 52581519, 47058645], + "base": 47058645, + "total_net": 870376156 + }, + "queries": [ + { + "query": "neword100", + "net": 870376156, + "runs": [956960680, 918670439, 917434801] + } + ] +} diff --git a/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.md b/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.md new file mode 100644 index 0000000..d0f2fa9 --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-instructions-tpcc-postgresql-smoke.md @@ -0,0 +1,40 @@ +# instructions-tpcc-postgresql on server3, 2026-10-07 + +**This is a smoke run.** It shows that the driver works. It ran on a small scale or on a shared machine, so its numbers are not baselines and no claim may use them. + +| Field | Value | +|---|---| +| suite | instructions-tpcc-postgresql | +| machine | server3 | +| date | 2026-10-07 | +| commit | b41f7e63 | +| smoke | true | +| note | smoke run: it shows that the runner works, it is not the ratchet | +| set | tpcc | +| engine_name | postgresql | +| queries_from | /root/work/fixed2/tpcc | +| repeat | 3 | +| kernel | 6.8.0-106-generic | + +## engine + +| Field | Value | +|---|---| +| mode | server cgroup | +| cgroup | system.slice/system-postgresql.slice/postgresql@19-ycsb.service | +| client | /usr/lib/postgresql/19/bin/psql, -X, -q, -v, ON_ERROR_STOP=1, -h, /var/run/postgresql, -p, 5433, -U, tpcc, -d, tpcc, -o, /dev/null, -f, - | + +## counts + +| Field | Value | +|---|---| +| base_runs | 52594897, 52581519, 47058645 | +| base | 47058645 | +| total_net | 870376156 | + +## queries + +| query | net | runs | +|---|---|---| +| neword100 | 870376156 | 956960680, 918670439, 917434801 | + diff --git a/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.json b/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.json new file mode 100644 index 0000000..19aac25 --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.json @@ -0,0 +1,48 @@ +{ + "suite": "instructions-ycsb-postgresql", + "machine": "server3", + "date": "2026-10-07", + "commit": "b41f7e63", + "smoke": true, + "note": "smoke run: it shows that the runner works, it is not the ratchet", + "set": "ycsb", + "engine_name": "postgresql", + "queries_from": "/root/work/fixed2/ycsb", + "repeat": 3, + "kernel": "6.8.0-106-generic", + "engine": { + "mode": "server cgroup", + "cgroup": "system.slice/system-postgresql.slice/postgresql@19-ycsb.service", + "client": [ + "/usr/lib/postgresql/19/bin/psql", + "-X", + "-q", + "-v", + "ON_ERROR_STOP=1", + "-h", + "/var/run/postgresql", + "-p", + "5433", + "-U", + "bench", + "-d", + "bench", + "-o", + "/dev/null", + "-f", + "-" + ] + }, + "counts": { + "base_runs": [11591789, 8824882, 9880131], + "base": 8824882, + "total_net": 197406645 + }, + "queries": [ + { + "query": "read1000", + "net": 197406645, + "runs": [830706694, 206231527, 238045194] + } + ] +} diff --git a/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.md b/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.md new file mode 100644 index 0000000..238dd82 --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-instructions-ycsb-postgresql-smoke.md @@ -0,0 +1,40 @@ +# instructions-ycsb-postgresql on server3, 2026-10-07 + +**This is a smoke run.** It shows that the driver works. It ran on a small scale or on a shared machine, so its numbers are not baselines and no claim may use them. + +| Field | Value | +|---|---| +| suite | instructions-ycsb-postgresql | +| machine | server3 | +| date | 2026-10-07 | +| commit | b41f7e63 | +| smoke | true | +| note | smoke run: it shows that the runner works, it is not the ratchet | +| set | ycsb | +| engine_name | postgresql | +| queries_from | /root/work/fixed2/ycsb | +| repeat | 3 | +| kernel | 6.8.0-106-generic | + +## engine + +| Field | Value | +|---|---| +| mode | server cgroup | +| cgroup | system.slice/system-postgresql.slice/postgresql@19-ycsb.service | +| client | /usr/lib/postgresql/19/bin/psql, -X, -q, -v, ON_ERROR_STOP=1, -h, /var/run/postgresql, -p, 5433, -U, bench, -d, bench, -o, /dev/null, -f, - | + +## counts + +| Field | Value | +|---|---| +| base_runs | 11591789, 8824882, 9880131 | +| base | 8824882 | +| total_net | 197406645 | + +## queries + +| query | net | runs | +|---|---|---| +| read1000 | 197406645 | 830706694, 206231527, 238045194 | + diff --git a/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.json b/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.json new file mode 100644 index 0000000..c14eb8b --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.json @@ -0,0 +1,553 @@ +{ + "suite": "tpcc-postgresql", + "machine": "server3", + "date": "2026-10-07", + "commit": "b41f7e63", + "smoke": true, + "note": "smoke run: it shows that the driver works, it is not a baseline", + "engine_name": "postgresql", + "cgroup": "/sys/fs/cgroup/system.slice/system-postgresql.slice/postgresql@19-ycsb.service", + "hammerdb_version": "6.0", + "rampup_min": 1, + "duration_min": 1, + "seed": 1, + "mix": "HammerDB: New-Order 10/23, Payment 10/23, Delivery, Stock-Level and Order-Status 1/23 each, no keying or think time", + "kernel": "6.8.0-106-generic", + "server_version": "19beta4", + "build": { + "warehouses": 2, + "virtual_users": 2, + "secs": 298.9, + "database_bytes": 227910159, + "server": { + "wall_s": 298.905259347, + "cpu_usec": 29346675, + "user_usec": 22771921, + "system_usec": 6574755, + "rbytes": 9531392, + "wbytes": 480669696, + "memory_peak_bytes": 665829376, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 607207424, + "pss_max_bytes": 212278272, + "memory_current_max_bytes": 665518080, + "samples": 1002 + } + }, + "warehouses": 2, + "check_before": [ + { + "condition": 1, + "text": "W_YTD = sum(D_YTD) for each warehouse", + "checked": 2, + "failed": 0 + }, + { + "condition": 2, + "text": "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 3, + "text": "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 4, + "text": "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "checked": 20, + "failed": 0 + } + ], + "fdatasync": { + "dir": "/var/lib/postgresql/19", + "rounds": 200, + "write_bytes": 8192, + "p50_us": 1674.7 + }, + "idle_base": { + "wall_s": 10.0091953, + "cpu_usec": 119402, + "user_usec": 41675, + "system_usec": 77728, + "rbytes": 0, + "wbytes": 3547136, + "memory_peak_bytes": 665829376, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 611614720, + "pss_max_bytes": 174551040, + "memory_current_max_bytes": 643317760, + "samples": 58 + }, + "server_cores": 8, + "rows": [ + { + "form": "procedures", + "virtual_users": 4, + "synchronous_commit": "on", + "synchronous_commit_set_by_driver": true, + "tpm_source": "HammerDB TEST RESULT: pg_stat_database commits and rollbacks", + "time_profile": { + "neword": { + "calls": 3745, + "min_ms": 2.114, + "avg_ms": 57.281, + "max_ms": 890.381, + "total_ms": 214516.204, + "p99_ms": 276.326, + "p95_ms": 167.94, + "p75_ms": 78.787, + "p50_ms": 41.078, + "p25_ms": 16.725, + "sd_ms": 59.783, + "ratio_percent": 44.546 + }, + "payment": { + "calls": 3735, + "min_ms": 0.935, + "avg_ms": 51.736, + "max_ms": 846.372, + "total_ms": 193234.171, + "p99_ms": 259.351, + "p95_ms": 163.743, + "p75_ms": 70.398, + "p50_ms": 32.316, + "p25_ms": 13.29, + "sd_ms": 58.181, + "ratio_percent": 40.126 + }, + "slev": { + "calls": 390, + "min_ms": 2.17, + "avg_ms": 110.641, + "max_ms": 678.323, + "total_ms": 43149.914, + "p99_ms": 457.944, + "p95_ms": 292.36, + "p75_ms": 156.867, + "p50_ms": 86.413, + "p25_ms": 35.935, + "sd_ms": 100.594, + "ratio_percent": 8.96 + }, + "delivery": { + "calls": 368, + "min_ms": 4.754, + "avg_ms": 66.163, + "max_ms": 374.149, + "total_ms": 24348.011, + "p99_ms": 287.993, + "p95_ms": 199.245, + "p75_ms": 83.917, + "p50_ms": 50.893, + "p25_ms": 20.743, + "sd_ms": 62.468, + "ratio_percent": 5.056 + }, + "ostat": { + "calls": 402, + "min_ms": 0.532, + "avg_ms": 14.584, + "max_ms": 383.284, + "total_ms": 5862.847, + "p99_ms": 124.195, + "p95_ms": 60.844, + "p75_ms": 11.774, + "p50_ms": 4.558, + "p25_ms": 1.386, + "sd_ms": 30.617, + "ratio_percent": 1.217 + } + }, + "time_profile_note": "HammerDB times each procedure call over the whole run, the ramp too", + "nopm": 2069, + "tpm": 4726, + "nopm_per_core": 259, + "server_cpu_usec_per_new_order": 8084, + "server_write_bytes_per_new_order": 56912, + "server_memory_peak": 796893184, + "sync_bound_nopm": 64491, + "server": { + "wall_s": 60.410309964, + "cpu_usec": 16724964, + "user_usec": 13846313, + "system_usec": 2878651, + "rbytes": 1191936, + "wbytes": 117751808, + "memory_peak_bytes": 796893184, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 737177600, + "pss_max_bytes": 202209280, + "memory_current_max_bytes": 796381184, + "samples": 187 + }, + "check": [ + { + "condition": 1, + "text": "W_YTD = sum(D_YTD) for each warehouse", + "checked": 2, + "failed": 0 + }, + { + "condition": 2, + "text": "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 3, + "text": "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 4, + "text": "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "checked": 20, + "failed": 0 + } + ] + }, + { + "form": "statements", + "virtual_users": 4, + "synchronous_commit": "on", + "synchronous_commit_set_by_driver": true, + "tpm_source": "the transactions that the terminals finished in the interval", + "latency": { + "neword": { + "count": 1102, + "mean_us": 142921.9, + "p50_us": 70778.9, + "p95_us": 490733.6, + "p99_us": 620757, + "p999_us": 1061158.9, + "max_us": 1384812.8 + }, + "payment": { + "count": 1014, + "mean_us": 52004.1, + "p50_us": 25034.8, + "p95_us": 181403.6, + "p99_us": 327155.7, + "p999_us": 501219.3, + "max_us": 560985.6 + }, + "ostat": { + "count": 86, + "mean_us": 13390.5, + "p50_us": 4816.9, + "p95_us": 63176.7, + "p99_us": 149141.9, + "p999_us": 149141.9, + "max_us": 149141.9 + }, + "delivery": { + "count": 117, + "mean_us": 218326.1, + "p50_us": 149946.4, + "p95_us": 637534.2, + "p99_us": 750780.4, + "p999_us": 1124841.1, + "max_us": 1124841.1 + }, + "slev": { + "count": 91, + "mean_us": 22687.6, + "p50_us": 10813.4, + "p95_us": 77594.6, + "p99_us": 153501.9, + "p999_us": 153501.9, + "max_us": 153501.9 + } + }, + "new_order_rollbacks": 9, + "deliveries_skipped_districts": 16, + "retries": 0, + "errors": 0, + "first_error": null, + "nopm": 1094, + "tpm": 2404, + "nopm_per_core": 137, + "server_cpu_usec_per_new_order": 11310, + "server_write_bytes_per_new_order": 59465, + "server_memory_peak": 796893184, + "sync_bound_nopm": 64491, + "server": { + "wall_s": 59.945986009, + "cpu_usec": 12406810, + "user_usec": 8426527, + "system_usec": 3980283, + "rbytes": 1654784, + "wbytes": 65232896, + "memory_peak_bytes": 796893184, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 675782656, + "pss_max_bytes": 190953472, + "memory_current_max_bytes": 726523904, + "samples": 147 + }, + "check": [ + { + "condition": 1, + "text": "W_YTD = sum(D_YTD) for each warehouse", + "checked": 2, + "failed": 0 + }, + { + "condition": 2, + "text": "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 3, + "text": "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 4, + "text": "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "checked": 20, + "failed": 0 + } + ] + }, + { + "form": "procedures", + "virtual_users": 4, + "synchronous_commit": "off", + "synchronous_commit_set_by_driver": true, + "tpm_source": "HammerDB TEST RESULT: pg_stat_database commits and rollbacks", + "time_profile": { + "neword": { + "calls": 6484, + "min_ms": 0.66, + "avg_ms": 34.181, + "max_ms": 954.056, + "total_ms": 221628.359, + "p99_ms": 250.179, + "p95_ms": 112.984, + "p75_ms": 46.524, + "p50_ms": 14.279, + "p25_ms": 5.283, + "sd_ms": 53.653, + "ratio_percent": 45.891 + }, + "payment": { + "calls": 6318, + "min_ms": 0.335, + "avg_ms": 24.118, + "max_ms": 854.833, + "total_ms": 152379.992, + "p99_ms": 191.788, + "p95_ms": 89.438, + "p75_ms": 28.322, + "p50_ms": 9.214, + "p25_ms": 3.12, + "sd_ms": 41.463, + "ratio_percent": 31.553 + }, + "slev": { + "calls": 641, + "min_ms": 1.591, + "avg_ms": 68.629, + "max_ms": 2118.375, + "total_ms": 43991.472, + "p99_ms": 563.597, + "p95_ms": 258.853, + "p75_ms": 74.916, + "p50_ms": 27.085, + "p25_ms": 6.692, + "sd_ms": 142.447, + "ratio_percent": 9.109 + }, + "delivery": { + "calls": 671, + "min_ms": 1.685, + "avg_ms": 58.793, + "max_ms": 1000.77, + "total_ms": 39450.339, + "p99_ms": 357.977, + "p95_ms": 200.71, + "p75_ms": 75.23, + "p50_ms": 31.834, + "p25_ms": 10.857, + "sd_ms": 83.027, + "ratio_percent": 8.169 + }, + "ostat": { + "calls": 634, + "min_ms": 0.299, + "avg_ms": 34.293, + "max_ms": 600.456, + "total_ms": 21741.885, + "p99_ms": 278.399, + "p95_ms": 115.99, + "p75_ms": 43.568, + "p50_ms": 14.04, + "p25_ms": 4.477, + "sd_ms": 57.434, + "ratio_percent": 4.502 + } + }, + "time_profile_note": "HammerDB times each procedure call over the whole run, the ramp too", + "nopm": 3770, + "tpm": 8575, + "nopm_per_core": 471, + "server_cpu_usec_per_new_order": 6172, + "server_write_bytes_per_new_order": 48765, + "server_memory_peak": 796893184, + "sync_bound_nopm": null, + "server": { + "wall_s": 60.789189832, + "cpu_usec": 23269569, + "user_usec": 19612017, + "system_usec": 3657552, + "rbytes": 22171648, + "wbytes": 183844864, + "memory_peak_bytes": 796893184, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 399056896, + "pss_max_bytes": 236539904, + "memory_current_max_bytes": 482545664, + "samples": 177 + }, + "check": [ + { + "condition": 1, + "text": "W_YTD = sum(D_YTD) for each warehouse", + "checked": 2, + "failed": 0 + }, + { + "condition": 2, + "text": "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 3, + "text": "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 4, + "text": "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "checked": 20, + "failed": 0 + } + ] + }, + { + "form": "statements", + "virtual_users": 4, + "synchronous_commit": "off", + "synchronous_commit_set_by_driver": true, + "tpm_source": "the transactions that the terminals finished in the interval", + "latency": { + "neword": { + "count": 1119, + "mean_us": 148849, + "p50_us": 51380.2, + "p95_us": 499122.2, + "p99_us": 796917.8, + "p999_us": 1090519, + "max_us": 1404995.1 + }, + "payment": { + "count": 1026, + "mean_us": 39874.5, + "p50_us": 7995.4, + "p95_us": 159383.6, + "p99_us": 362807.3, + "p999_us": 692060.2, + "max_us": 821887.2 + }, + "ostat": { + "count": 94, + "mean_us": 17036.6, + "p50_us": 3424.3, + "p95_us": 76021.8, + "p99_us": 164448.3, + "p999_us": 164448.3, + "max_us": 164448.3 + }, + "delivery": { + "count": 117, + "mean_us": 225826.6, + "p50_us": 116916.2, + "p95_us": 729808.9, + "p99_us": 977272.8, + "p999_us": 1301864, + "max_us": 1301864 + }, + "slev": { + "count": 95, + "mean_us": 31930.9, + "p50_us": 14680.1, + "p95_us": 108003.3, + "p99_us": 171355, + "p999_us": 171355, + "max_us": 171355 + } + }, + "new_order_rollbacks": 12, + "deliveries_skipped_districts": 17, + "retries": 0, + "errors": 0, + "first_error": null, + "nopm": 1109, + "tpm": 2449, + "nopm_per_core": 139, + "server_cpu_usec_per_new_order": 10920, + "server_write_bytes_per_new_order": 35351, + "server_memory_peak": 796893184, + "sync_bound_nopm": null, + "server": { + "wall_s": 59.973384063, + "cpu_usec": 12121626, + "user_usec": 8961422, + "system_usec": 3160205, + "rbytes": 1486848, + "wbytes": 39239680, + "memory_peak_bytes": 796893184, + "memory_peak_scope": "since the cgroup was created", + "file_max_bytes": 384950272, + "pss_max_bytes": 192243712, + "memory_current_max_bytes": 428191744, + "samples": 124 + }, + "check": [ + { + "condition": 1, + "text": "W_YTD = sum(D_YTD) for each warehouse", + "checked": 2, + "failed": 0 + }, + { + "condition": 2, + "text": "D_NEXT_O_ID - 1 = max(O_ID) = max(NO_O_ID) for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 3, + "text": "max(NO_O_ID) - min(NO_O_ID) + 1 = rows in NEW-ORDER for each district", + "checked": 20, + "failed": 0 + }, + { + "condition": 4, + "text": "sum(O_OL_CNT) = rows in ORDER-LINE for each district", + "checked": 20, + "failed": 0 + } + ] + } + ] +} diff --git a/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.md b/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.md new file mode 100644 index 0000000..3084f40 --- /dev/null +++ b/reports/2026-10-07/b41f7e63-server3-tpcc-postgresql-smoke.md @@ -0,0 +1,89 @@ +# tpcc-postgresql on server3, 2026-10-07 + +**This is a smoke run.** It shows that the driver works. It ran on a small scale or on a shared machine, so its numbers are not baselines and no claim may use them. + +| Field | Value | +|---|---| +| suite | tpcc-postgresql | +| machine | server3 | +| date | 2026-10-07 | +| commit | b41f7e63 | +| smoke | true | +| note | smoke run: it shows that the driver works, it is not a baseline | +| engine_name | postgresql | +| cgroup | /sys/fs/cgroup/system.slice/system-postgresql.slice/postgresql@19-ycsb.service | +| hammerdb_version | 6.0 | +| rampup_min | 1 | +| duration_min | 1 | +| seed | 1 | +| mix | HammerDB: New-Order 10/23, Payment 10/23, Delivery, Stock-Level and Order-Status 1/23 each, no keying or think time | +| kernel | 6.8.0-106-generic | +| server_version | 19beta4 | +| warehouses | 2 | +| server_cores | 8 | + +## build + +| Field | Value | +|---|---| +| warehouses | 2 | +| virtual_users | 2 | +| secs | 298.9 | +| database_bytes | 227910159 | +| server.wall_s | 298.905259 | +| server.cpu_usec | 29346675 | +| server.user_usec | 22771921 | +| server.system_usec | 6574755 | +| server.rbytes | 9531392 | +| server.wbytes | 480669696 | +| server.memory_peak_bytes | 665829376 | +| server.memory_peak_scope | since the cgroup was created | +| server.file_max_bytes | 607207424 | +| server.pss_max_bytes | 212278272 | +| server.memory_current_max_bytes | 665518080 | +| server.samples | 1002 | + +## check_before + +| condition | text | checked | failed | +|---|---|---|---| +| 1 | W_YTD = sum(D_YTD) for each warehouse | 2 | 0 | +| 2 | | 20 | 0 | +| 3 | | 20 | 0 | +| 4 | sum(O_OL_CNT) = rows in ORDER-LINE for each district | 20 | 0 | + +## fdatasync + +| Field | Value | +|---|---| +| dir | /var/lib/postgresql/19 | +| rounds | 200 | +| write_bytes | 8192 | +| p50_us | 1674.7 | + +## idle_base + +| Field | Value | +|---|---| +| wall_s | 10.009195 | +| cpu_usec | 119402 | +| user_usec | 41675 | +| system_usec | 77728 | +| rbytes | 0 | +| wbytes | 3547136 | +| memory_peak_bytes | 665829376 | +| memory_peak_scope | since the cgroup was created | +| file_max_bytes | 611614720 | +| pss_max_bytes | 174551040 | +| memory_current_max_bytes | 643317760 | +| samples | 58 | + +## rows + +| form | virtual_users | synchronous_commit | synchronous_commit_set_by_driver | tpm_source | time_profile.neword.calls | time_profile.neword.min_ms | time_profile.neword.avg_ms | time_profile.neword.max_ms | time_profile.neword.total_ms | time_profile.neword.p99_ms | time_profile.neword.p95_ms | time_profile.neword.p75_ms | time_profile.neword.p50_ms | time_profile.neword.p25_ms | time_profile.neword.sd_ms | time_profile.neword.ratio_percent | time_profile.payment.calls | time_profile.payment.min_ms | time_profile.payment.avg_ms | time_profile.payment.max_ms | time_profile.payment.total_ms | time_profile.payment.p99_ms | time_profile.payment.p95_ms | time_profile.payment.p75_ms | time_profile.payment.p50_ms | time_profile.payment.p25_ms | time_profile.payment.sd_ms | time_profile.payment.ratio_percent | time_profile.slev.calls | time_profile.slev.min_ms | time_profile.slev.avg_ms | time_profile.slev.max_ms | time_profile.slev.total_ms | time_profile.slev.p99_ms | time_profile.slev.p95_ms | time_profile.slev.p75_ms | time_profile.slev.p50_ms | time_profile.slev.p25_ms | time_profile.slev.sd_ms | time_profile.slev.ratio_percent | time_profile.delivery.calls | time_profile.delivery.min_ms | time_profile.delivery.avg_ms | time_profile.delivery.max_ms | time_profile.delivery.total_ms | time_profile.delivery.p99_ms | time_profile.delivery.p95_ms | time_profile.delivery.p75_ms | time_profile.delivery.p50_ms | time_profile.delivery.p25_ms | time_profile.delivery.sd_ms | time_profile.delivery.ratio_percent | time_profile.ostat.calls | time_profile.ostat.min_ms | time_profile.ostat.avg_ms | time_profile.ostat.max_ms | time_profile.ostat.total_ms | time_profile.ostat.p99_ms | time_profile.ostat.p95_ms | time_profile.ostat.p75_ms | time_profile.ostat.p50_ms | time_profile.ostat.p25_ms | time_profile.ostat.sd_ms | time_profile.ostat.ratio_percent | time_profile_note | nopm | tpm | nopm_per_core | server_cpu_usec_per_new_order | server_write_bytes_per_new_order | server_memory_peak | sync_bound_nopm | server.wall_s | server.cpu_usec | server.user_usec | server.system_usec | server.rbytes | server.wbytes | server.memory_peak_bytes | server.memory_peak_scope | server.file_max_bytes | server.pss_max_bytes | server.memory_current_max_bytes | server.samples | check.0.condition | check.0.text | check.0.checked | check.0.failed | check.1.condition | check.1.checked | check.1.failed | check.2.condition | check.2.checked | check.2.failed | check.3.condition | check.3.text | check.3.checked | check.3.failed | latency.neword.count | latency.neword.mean_us | latency.neword.p50_us | latency.neword.p95_us | latency.neword.p99_us | latency.neword.p999_us | latency.neword.max_us | latency.payment.count | latency.payment.mean_us | latency.payment.p50_us | latency.payment.p95_us | latency.payment.p99_us | latency.payment.p999_us | latency.payment.max_us | latency.ostat.count | latency.ostat.mean_us | latency.ostat.p50_us | latency.ostat.p95_us | latency.ostat.p99_us | latency.ostat.p999_us | latency.ostat.max_us | latency.delivery.count | latency.delivery.mean_us | latency.delivery.p50_us | latency.delivery.p95_us | latency.delivery.p99_us | latency.delivery.p999_us | latency.delivery.max_us | latency.slev.count | latency.slev.mean_us | latency.slev.p50_us | latency.slev.p95_us | latency.slev.p99_us | latency.slev.p999_us | latency.slev.max_us | new_order_rollbacks | deliveries_skipped_districts | retries | errors | first_error | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| procedures | 4 | on | true | HammerDB TEST RESULT: pg_stat_database commits and rollbacks | 3745 | 2.114 | 57.281 | 890.381 | 214516.204 | 276.326 | 167.94 | 78.787 | 41.078 | 16.725 | 59.783 | 44.546 | 3735 | 0.935 | 51.736 | 846.372 | 193234.171 | 259.351 | 163.743 | 70.398 | 32.316 | 13.29 | 58.181 | 40.126 | 390 | 2.17 | 110.641 | 678.323 | 43149.914 | 457.944 | 292.36 | 156.867 | 86.413 | 35.935 | 100.594 | 8.96 | 368 | 4.754 | 66.163 | 374.149 | 24348.011 | 287.993 | 199.245 | 83.917 | 50.893 | 20.743 | 62.468 | 5.056 | 402 | 0.532 | 14.584 | 383.284 | 5862.847 | 124.195 | 60.844 | 11.774 | 4.558 | 1.386 | 30.617 | 1.217 | HammerDB times each procedure call over the whole run, the ramp too | 2069 | 4726 | 259 | 8084 | 56912 | 796893184 | 64491 | 60.41031 | 16724964 | 13846313 | 2878651 | 1191936 | 117751808 | 796893184 | since the cgroup was created | 737177600 | 202209280 | 796381184 | 187 | 1 | W_YTD = sum(D_YTD) for each warehouse | 2 | 0 | 2 | 20 | 0 | 3 | 20 | 0 | 4 | sum(O_OL_CNT) = rows in ORDER-LINE for each district | 20 | 0 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| statements | 4 | on | true | the transactions that the terminals finished in the interval | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1094 | 2404 | 137 | 11310 | 59465 | 796893184 | 64491 | 59.945986 | 12406810 | 8426527 | 3980283 | 1654784 | 65232896 | 796893184 | since the cgroup was created | 675782656 | 190953472 | 726523904 | 147 | 1 | W_YTD = sum(D_YTD) for each warehouse | 2 | 0 | 2 | 20 | 0 | 3 | 20 | 0 | 4 | sum(O_OL_CNT) = rows in ORDER-LINE for each district | 20 | 0 | 1102 | 142921.9 | 70778.9 | 490733.6 | 620757 | 1061158.9 | 1384812.8 | 1014 | 52004.1 | 25034.8 | 181403.6 | 327155.7 | 501219.3 | 560985.6 | 86 | 13390.5 | 4816.9 | 63176.7 | 149141.9 | 149141.9 | 149141.9 | 117 | 218326.1 | 149946.4 | 637534.2 | 750780.4 | 1124841.1 | 1124841.1 | 91 | 22687.6 | 10813.4 | 77594.6 | 153501.9 | 153501.9 | 153501.9 | 9 | 16 | 0 | 0 | null | +| procedures | 4 | off | true | HammerDB TEST RESULT: pg_stat_database commits and rollbacks | 6484 | 0.66 | 34.181 | 954.056 | 221628.359 | 250.179 | 112.984 | 46.524 | 14.279 | 5.283 | 53.653 | 45.891 | 6318 | 0.335 | 24.118 | 854.833 | 152379.992 | 191.788 | 89.438 | 28.322 | 9.214 | 3.12 | 41.463 | 31.553 | 641 | 1.591 | 68.629 | 2118.375 | 43991.472 | 563.597 | 258.853 | 74.916 | 27.085 | 6.692 | 142.447 | 9.109 | 671 | 1.685 | 58.793 | 1000.77 | 39450.339 | 357.977 | 200.71 | 75.23 | 31.834 | 10.857 | 83.027 | 8.169 | 634 | 0.299 | 34.293 | 600.456 | 21741.885 | 278.399 | 115.99 | 43.568 | 14.04 | 4.477 | 57.434 | 4.502 | HammerDB times each procedure call over the whole run, the ramp too | 3770 | 8575 | 471 | 6172 | 48765 | 796893184 | null | 60.78919 | 23269569 | 19612017 | 3657552 | 22171648 | 183844864 | 796893184 | since the cgroup was created | 399056896 | 236539904 | 482545664 | 177 | 1 | W_YTD = sum(D_YTD) for each warehouse | 2 | 0 | 2 | 20 | 0 | 3 | 20 | 0 | 4 | sum(O_OL_CNT) = rows in ORDER-LINE for each district | 20 | 0 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| statements | 4 | off | true | the transactions that the terminals finished in the interval | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1109 | 2449 | 139 | 10920 | 35351 | 796893184 | null | 59.973384 | 12121626 | 8961422 | 3160205 | 1486848 | 39239680 | 796893184 | since the cgroup was created | 384950272 | 192243712 | 428191744 | 124 | 1 | W_YTD = sum(D_YTD) for each warehouse | 2 | 0 | 2 | 20 | 0 | 3 | 20 | 0 | 4 | sum(O_OL_CNT) = rows in ORDER-LINE for each district | 20 | 0 | 1119 | 148849 | 51380.2 | 499122.2 | 796917.8 | 1090519 | 1404995.1 | 1026 | 39874.5 | 7995.4 | 159383.6 | 362807.3 | 692060.2 | 821887.2 | 94 | 17036.6 | 3424.3 | 76021.8 | 164448.3 | 164448.3 | 164448.3 | 117 | 225826.6 | 116916.2 | 729808.9 | 977272.8 | 1301864 | 1301864 | 95 | 31930.9 | 14680.1 | 108003.3 | 171355 | 171355 | 171355 | 12 | 17 | 0 | 0 | null | +