Clean up documentation

docs/CLIENT_PERFORMANCE_TUNING.md

@@ -32,12 +32,12 @@ enable_cpu_affinity: true # Auto-compute NUMA-aware plan (default)
 # enable_cpu_affinity: false  # Disabled
 ```
 
-**Auto mode allocation** (default 6 physical cores for loadgen):
+**Auto mode allocation** (default 5 physical cores for loadgen, `DEFAULT_LOADGEN_CORES`):
 
 - 1 core: Session thread (scheduler, busy-wait timing)
 - 1 core: Event loop thread (uvloop, response handling)
-- 4 cores: ZMQ I/O threads
-- Remaining physical cores: Workers (one per core with all SMT siblings)
+- Remaining cores: ZMQ I/O threads (up to 4, sharing the leftover loadgen cores)
+- All other physical cores: Workers (one per core with all SMT siblings)
 
 ## Platform Notes
 

docs/PERF_ARCHITECTURE.md

@@ -367,10 +367,12 @@ Workers are mostly I/O-bound (waiting on HTTP responses).
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
-| Component   | Allocation                       | Rationale                                                                 |
-| ----------- | -------------------------------- | ------------------------------------------------------------------------- |
-| **LoadGen** | 2 physical cores (4 logical)     | Session thread + event loop thread. Bottleneck - gets fastest cores.      |
-| **Workers** | 1 physical core each (2 logical) | I/O-bound. Full core isolation prevents context switches, reduces jitter. |
+| Component   | Allocation                                 | Rationale                                                                                      |
+| ----------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------- |
+| **LoadGen** | `DEFAULT_LOADGEN_CORES` = 5 physical cores | Session thread + event loop thread + up to 4 ZMQ I/O threads. Bottleneck - gets fastest cores. |
+| **Workers** | 1 physical core each (2 logical)           | I/O-bound. Full core isolation prevents context switches, reduces jitter.                      |
+
+> The core-ordering diagram above is schematic — it shows two `LG` cells for illustration; the actual default reserves 5 physical cores for loadgen (`DEFAULT_LOADGEN_CORES` in `cpu_affinity.py`).
 
 ### Why Both Hyperthreads Per Core
 

docs/async_utils/DESIGN.md

@@ -69,17 +69,18 @@ short-lived coroutines on the hot path.
 
 ### `EventPublisherService`
 
-Singleton via `SingletonMixin` — after the first construction, subsequent calls return the
-cached instance. The first construction requires a `ManagedZMQContext`. Subscribers receive
-`EventRecord` messages over a ZMQ SUB socket.
+A plain per-instance class wrapping `ZmqMessagePublisher[EventRecord]` with `LoopManager`
+integration and auto-generated socket names. Each construction requires a `ManagedZMQContext`.
+Subscribers receive `EventRecord` messages over a ZMQ SUB socket.
 
 ```python
-class EventPublisherService(SingletonMixin, ZmqEventRecordPublisher):
+class EventPublisherService(ZmqMessagePublisher[EventRecord]):
     def __init__(
         self,
         managed_zmq_context: ManagedZMQContext,
         extra_eager: bool = False,
         isolated_event_loop: bool = False,
+        send_threshold: int = 1000,
     ) -> None
 
     def publish(self, record: EventRecord) -> None

docs/async_utils/services/event_logger/DESIGN.md

@@ -212,9 +212,9 @@ usage: uv run python -m inference_endpoint.async_utils.services.event_logger
 | `--socket-name` | Yes      | —       | Socket name within socket-dir                             |
 | `--writers`     | No       | `jsonl` | Writer backends: `jsonl`, `sql`, or both                  |
 
-## Not Yet Wired
+## Wiring
 
-The EventRecord pub/sub infrastructure is ready, but actual `publish(EventRecord(...))`
-calls have not been connected in the load generator or worker processes. Once wired,
-the event logger will receive and persist all session/sample/error events published
-during a benchmark run.
+`publish(EventRecord(...))` calls are connected in the load generator (`BenchmarkSession`
+publishes STARTED / ISSUED / COMPLETE / ERROR events via its `EventPublisher`; the prompt rides on
+the `ISSUED` event's `PromptData` payload rather than a separate event), so the event
+logger receives and persists all session/sample/error events published during a benchmark run.

docs/async_utils/transport/zmq/ready_check_design.md

@@ -40,7 +40,7 @@ This means one receiver socket handles readiness from all subprocesses.
 - Binds a ZMQ PULL socket on an IPC path
 - `wait(timeout)` blocks until `count` signals arrive
 - Returns list of identities in arrival order
-- Closes socket after all signals received (or on timeout)
+- Closes the socket after all signals are received, but deliberately **not** on timeout (the caller may retry)
 - Timeout is a total deadline, not per-message
 
 ### `send_ready_signal()` (subprocess side)

docs/compliance_audit_plan.md

@@ -182,7 +182,7 @@ report is already written).
         ╔═══════════ for each spec (back-to-back) ═════════════╗         │
         ║                    ▼                                  ║         │
         ║   ┌──────────────────────────────────────────┐       ║         │
-        ║   │ phase_cfg = perf-only (accuracy datasets  │       ║         │
+        ║   │ phase_cfg = perf-only* (accuracy datasets │       ║         │
         ║   │   dropped; audit=None; report_dir=<label>)│       ║         │
         ║   │ ctx = setup_benchmark(phase_cfg,          │       ║         │
         ║   │   audit_run_spec)                         │       ║         │
@@ -241,6 +241,11 @@ report is already written).
             └──────────────────────────────────────────┘
 ```
 
+\* Perf-only is the **default**. A phase may set `AuditRunSpec.test_mode` (`ACC`/`BOTH`) to keep
+its accuracy datasets instead of dropping them; the orchestrator reads `spec.test_mode` per phase
+rather than hardcoding perf-only. This override is supported but currently **unused** — every
+registered audit (TEST04) leaves `test_mode` at its `PERF` default.
+
 The first-phase gate calls `AuditTest.validate(cfg, N, load_pattern)` once `N` (the loaded
 dataset size) is known: each audit owns its own preconditions there, so the generic loop
 never encodes a single test's rules. For the output-caching test that means only
@@ -388,8 +393,11 @@ The generic loop never names a specific test:
    phase issues load.
 4. Execute each spec back-to-back via the existing `setup_benchmark` /
    `run_benchmark_async` path (no duplicated report-dir or `config.yaml` logic). Each phase
-   config is performance-only (accuracy datasets dropped so no phase re-issues or re-scores
-   them) and has `audit=None` to prevent re-entry into `run_audit`. If any phase raises
+   config is performance-only **by default** (accuracy datasets dropped so no phase re-issues
+   or re-scores them) and has `audit=None` to prevent re-entry into `run_audit`. A phase can
+   override this per-`AuditRunSpec` via `test_mode` (`ACC`/`BOTH` keeps its accuracy datasets);
+   the orchestrator reads `spec.test_mode` rather than hardcoding perf-only. This override is
+   supported but currently unused — every registered audit (TEST04) runs perf-only. If any phase raises
    (`SetupError` / `ExecutionError`), `run_audit` aborts **without verifying** — a crashed
    phase must never produce a result. A phase that returns but whose `Report.complete` is
    `False` (metrics drain timed out, or the run was interrupted → partial stats) is likewise

docs/core/DESIGN.md

@@ -52,7 +52,6 @@ Represents one SSE delta from a streaming response.
 | ---------------- | ---------------- | ------------------------------ |
 | `id`             | `str`            | Matches originating `Query.id` |
 | `response_chunk` | `str`            | Incremental token text         |
-| `is_complete`    | `bool`           | True for the final chunk       |
 | `metadata`       | `dict[str, Any]` | Per-chunk metadata             |
 
 ### `TextModelOutput`

docs/dataset_manager/DESIGN.md

@@ -77,8 +77,9 @@ parser/remap config, and dataset name. Format is inferred from file extension wh
 - `.parquet` → `PARQUET`
 - explicit `format=huggingface` → `HF`
 
-Presets (e.g. `"gpqa::Qwen/Qwen3-8B"`) are encoded in `config.name` as a `"::"` split — the
-factory resolves them to a predefined dataset class with a model-specific transform stack.
+Presets (e.g. `"gpqa::gptoss"`) are encoded in `config.name` as a `"::"` split — `<dataset>::<preset>` —
+where the factory resolves the first segment to a predefined dataset class and the second to a named
+preset with its transform stack.
 
 ### `Transform` (abstract base)
 
@@ -116,8 +117,8 @@ configs. Each predefined dataset ships with default transforms for supported mod
 
 ## Preset System
 
-A preset string like `"gpqa::Qwen/Qwen3-8B"` resolves to a predefined dataset with a
-model-specific transform stack pre-applied. This is used by rulesets to ensure consistent
+A preset string like `"gpqa::gptoss"` (`<dataset>::<preset>`) resolves to a predefined dataset with a
+named preset's transform stack pre-applied. This is used by rulesets to ensure consistent
 prompt formatting across submissions.
 
 ## Design Decisions

docs/evaluation/DESIGN.md

@@ -35,11 +35,12 @@ accuracy summary written into benchmark results
 
 ## Files
 
-| File             | Purpose                                                           |
-| ---------------- | ----------------------------------------------------------------- |
-| `extractor.py`   | Extracts model answer from raw text (regex, boxed-answer parsing) |
-| `scoring.py`     | Compares extracted answer to ground truth label                   |
-| `livecodebench/` | LiveCodeBench-specific code execution pipeline                    |
+| File             | Purpose                                                                                                                                      |
+| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
+| `extractor.py`   | Extracts model answer from raw text (regex, boxed-answer parsing)                                                                            |
+| `scoring.py`     | Compares extracted answer to ground truth label                                                                                              |
+| `bfcl_v4_*.py`   | BFCL v4 function-calling eval: `bfcl_v4_execution.py`, `bfcl_v4_scorer.py`, `bfcl_v4_metrics.py`, and the multi-turn `runner`/`scorer`/`cli` |
+| `livecodebench/` | LiveCodeBench-specific code execution pipeline                                                                                               |
 
 ## LiveCodeBench
 
@@ -60,13 +61,19 @@ Files:
 
 The scorer registry in `evaluation/scoring.py` currently includes:
 
-| Method                | Description                                                    |
-| --------------------- | -------------------------------------------------------------- |
-| `pass_at_1`           | Exact-match style scoring; also used by the LiveCodeBench path |
-| `string_match`        | Whitespace-trimmed string equality                             |
-| `rouge`               | ROUGE-based text generation scoring                            |
-| `code_bench_scorer`   | LiveCodeBench code-execution scoring                           |
-| `shopify_category_f1` | Shopify category F1 evaluation                                 |
+| Method                      | Description                                                    |
+| --------------------------- | -------------------------------------------------------------- |
+| `pass_at_1`                 | Exact-match style scoring; also used by the LiveCodeBench path |
+| `string_match`              | Whitespace-trimmed string equality                             |
+| `rouge`                     | ROUGE-based text generation scoring                            |
+| `code_bench_scorer`         | LiveCodeBench code-execution scoring                           |
+| `shopify_category_f1`       | Shopify category F1 evaluation                                 |
+| `agentic_inference_inline`  | Inline scoring for agentic multi-turn inference                |
+| `vbench`                    | VBench video-generation accuracy (WAN 2.2 T2V)                 |
+| `bfcl_v4`                   | BFCL v4 function-calling accuracy                              |
+| `legacy_mlperf_deepseek_r1` | MLPerf DeepSeek-R1 combined multi-subset accuracy              |
+
+The registry (`Scorer.PREDEFINED`) is auto-populated from `Scorer` subclasses via `__init_subclass__`.
 
 The scoring configuration used by benchmark execution is specified per accuracy dataset under
 `datasets[].accuracy_config`, including `accuracy_config.eval_method`,