perf: parallelize YCSB key preload

[perfloop]

Problem

YCSB preloaded keys one at a time. preloadKeys waited for each Put or PutStream to return before issuing the next write, so the configured concurrency did not apply to the load phase. The benchmark starts after preload, but command completion still paid the serial load cost.

Fix

Generate keys and values in one deterministic producer, then send bounded jobs to a worker pool sized by the configured concurrency. Workers share the existing client and perform writes concurrently, while a collector preserves completion accounting, progress updates, error handling, and the first-error ordering. All workers are joined before benchmark traffic starts.

The pool keeps only a bounded number of owned jobs in flight. It preserves the key/value mapping, later RNG state, streaming selection, cancellation behavior, and preload error threshold.

Workload: RunYCSBWithContext with 64 100-byte preload keys, 64 read operations, 8 workers, loopback gRPC, and a 1 ms PutObject handler delay

Metric	Before	After	Change
ns/op	150180727	29730331	80.2% lower
Checks: 2 passed.

Testing

• TestPreloadKeysPreservesValuesAndRNG
• TestPreloadKeysPreservesErrorThreshold
• TestPreloadKeysWaitsForWorkersOnCancellation
• make test-client TESTRUN=PreloadKeys
• make fmt-check

---

Generated by Perfloop. Measurements and checks.

---

Note

Low Risk
Benchmark-tooling change only; behavior is explicitly tested for parity with the serial preload (RNG, errors, cancellation).

Overview
Parallelizes YCSB key preload so the load phase respects Concurrency instead of issuing one put at a time.

preloadKeys now uses a single deterministic producer (same key/value generation order and post-preload RNG state), a bounded job queue sized to the worker count, and a pool of workers that perform plain/streaming/CAS puts concurrently. A collector goroutine tracks successes, errors, spinner progress (by completed puts), cancellation, and picks the lowest-sequence error so the 10% failure threshold and “first error” message match the old serial behavior. Cancellation messaging reports completed puts vs expectedPuts.

Tests add a mock server with injectable put failures, latency, and concurrency tracking, plus coverage for correctness (values + RNG), error threshold, clean shutdown on cancel, and a preload benchmark.

^{Reviewed by Cursor Bugbot for commit af7a3c6. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

The PR parallelizes YCSB key preloading through a bounded worker pool while retaining deterministic generation, ordered error reporting, and cancellation cleanup.

• Adds concurrent preload production, execution, and result collection.
• Adds tests for value and RNG preservation, failure thresholds, concurrency bounds, and cancellation.
• Adds a benchmark covering parallel preload performance.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
client/cmd/ycsb/ycsb.go	Replaces serial preload writes with a bounded worker pool and synchronized result collection; no eligible follow-up defect was established.
client/cmd/ycsb/ycsb_test.go	Adds focused preload concurrency, determinism, error-threshold, cancellation, and benchmark coverage without introducing a publishable follow-up issue.

_{Reviews (2): Last reviewed commit: "perf: parallelize YCSB key preload" | Re-trigger Greptile}

[ovaistariq]

Rebased onto main (af7a3c6). The preload function had been rewritten by #260 (a CAS namespace and puts-based accounting), so the worker pool was merged into that shape: one deterministic producer keeps the original key/value/RNG order across both namespaces, the pool does the puts, and the collector keeps the puts-based threshold. The three tests were ported onto main's CAS-aware test server and pass. The rebase squashed the PR's four commits into one, still authored and signed off by perfloop. Ready to merge.