test: assert delivered rate and concurrency match config

[Bslabe123]

Part of #633. Asserts for the first time that the load inference-perf actually offered matches the load it was configured with: achieved request rate against configured rate, and delivered in-flight concurrency against concurrency_level.

Row in #606: "Load-shape accuracy (#633)", sim tier.

Why this is needed

Every other e2e test checks the measurement side; nothing checks the stimulus side, which sweep silently depends on. docs/loadgen.md promises "the exact concurrency specified"; unit tests cover only the split arithmetic of _set_worker_concurrency, not whether workers honour it over real sockets. A semaphore bug throttling to concurrency_level - 1 passes everything today.

What the test does

• Faked: the server. llm-d-inference-sim with pinned per-request latencies. Deliberately sim: against real vLLM these assertions would measure the server's capacity, not the load generator.
• Oracle: the configured load, plus delivered rate and in-flight concurrency reconstructed in the new e2e/utils/load_shape.py from raw per-request start_time/end_time pairs, never from reportgen's summary numbers.
• Rate (constant and poisson, 600 requests at 40 qps): count and requested_rate exact; reported send_duration and achieved_rate must equal the recomputation (rel 1e-9, so any drift is a reportgen bug); delivered rate within a tolerance derived from the arrival process.
• Tolerance as a function of n, not a magic number. constant: gaps are rescaled to sum to exactly duration, so the only stochastic term is the first gap, Exp(1)/n relative, k = 12 for ~1e-5 false failure. poisson: renewal time, CoV 1/sqrt(n), 4 sigma. Both floored at 5% for jitter. At n=600: 5% and 16.3%; tightens by raising n, never by shrinking the multiplier.
• Concurrency (concurrency_level 8 and 5, 2 workers each, so one split leaves a remainder): in-flight is a sweep line over starts and ends, ties resolved ends-before-starts. The plateau window [C-th earliest start, latest start] excludes ramp-up and drain by construction (request k cannot start until k-C finished). Delivered concurrency must never exceed C (exact) and must be within half a slot of C time-weighted (measured handoff deficit on the sim is about 0.04 slot, so 10x headroom while still failing a whole-slot deficit).
• load_summary["concurrency"] is asserted only as a wiring check, labelled as one (reportgen copies it off the stage config). achieved_rate is not asserted on concurrent stages, which main.py rewrites to rate=num_requests, duration=1.
• Eight helper self-tests run without the sim and prove the assertions go red: C-1 and C+1 both raise, the tolerance function is checked at known values, the sweep line against hand-computed segments.

Per the row's lane note, dispatch scheduling arithmetic stays unit level in #659; only what needs real processes and sockets is asserted here.

Status

Stands alone on main (7bfbaed).

Verified locally with llm-d-inference-sim v0.6.1: 12 passed for the new module (4 sim-backed, 8 self-tests), 109s serial, 64s at -n 4. pdm run validate clean; unit suite unaffected (889 passed, 15 skipped). Kill-mutant: dropping the concurrency slack to 0.001 fails both cases.

Not yet verified: the CI runner. Tolerances have roughly 10x headroom over what was observed (constant 0.5% off a 5% budget, poisson 1.5% off 16.3%), but a merge-gating timing assertion earns its flake budget over repeated runs.

Limit of the oracle (in the module docstring): the timestamps are the client's own, so this catches a generator falling behind schedule, a semaphore admitting the wrong count, and a wrong reportgen rate, but not a timestamping bug. The #481 absorber records offered load server-side; once it lands this test can be retargeted at it without changing assertions.

Helpers live in e2e/utils/load_shape.py, as #633 asks, so the #632 worker matrix can reuse them.

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bslabe123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [Bslabe123]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment