Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions docs/benchmark-modes/timing-modes-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ AIPerf determines how to schedule requests based on which CLI options you specif
| `--concurrency` (alone) | Saturation/throughput testing | Send requests as fast as possible within concurrency limits |
| `--fixed-schedule` | Trace replay | Replay requests at exact timestamps from dataset |
| `--user-centric-rate` | KV cache benchmarking | Per-user rate limiting with consistent turn gaps |
| `--adaptive-scale` | SLA boundary discovery | Adapt one load-control variable during a duration-based phase, then sustain near the last passing boundary |

### Option Priority

Expand All @@ -28,7 +27,7 @@ When multiple options are specified, AIPerf uses this priority:
3. `--request-rate` → Rate-based scheduling with arrival patterns
4. `--concurrency` only → Burst mode (as fast as possible within limits)

`--adaptive-scale` wraps the selected duration-based profiling phase after normal scheduling is chosen. It adapts one control variable (`concurrency`, `prefill_concurrency`, `request_rate`, or `users`) and rejects fixed ramps on that same variable. See [Adaptive Scale](../tutorials/adaptive-scale.md) for examples and artifact semantics.
`adaptive_scale` is not a scheduling option. It is a YAML-only controller overlay that wraps a duration-based profiling phase after normal scheduling is chosen. It adapts one control variable (`concurrency`, `prefill_concurrency`, `request_rate`, or `users`) and rejects fixed ramps on that same variable. See [Adaptive Scale](../tutorials/adaptive-scale.md) for examples and artifact semantics.

---

Expand Down
74 changes: 0 additions & 74 deletions docs/cli-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -954,43 +954,6 @@ Duration in seconds to ramp prefill concurrency from 1 to target.
Duration in seconds to ramp request rate from a proportional minimum to target. Start rate is calculated as target * (update_interval / duration), ensuring correct behavior for target rates below 1 QPS. Useful for gradual warm-up of the target system.
<br/>_Constraints: > 0_

#### `--adaptive-scale`

Enable stable single-run adaptive scale control. Use --adaptive-scale-control variable:min,max:type to choose the controlled variable and bounds, and --adaptive-scale-sla metric:stat:op:threshold to define pass/fail criteria. Also requires --benchmark-duration and --adaptive-sustain-duration.
<br/>_Flag (no value required)_

#### `--adaptive-sustain-duration` `<float>`

Duration in seconds to sustain load near the discovered adaptive scale boundary.
<br/>_Constraints: > 0_

#### `--adaptive-assessment-period`, `--adaptive-scale-assessment-period` `<float>`

Duration in seconds for each adaptive scale SLA assessment window.
<br/>_Constraints: ≥ 1.0_

#### `--adaptive-scale-control` `<str>`

Compact adaptive scale control spec in variable:min,max:type form. The variable is one of concurrency, prefill_concurrency, request_rate, or users; min and max are required explicit bounds; type is int for discrete controls and float for rate controls. Examples: concurrency:1,1000:int, prefill_concurrency:1,8:int, request_rate:1,200:float, users:10,500:int. Do not combine this with expanded --adaptive-control-* flags.

#### `--adaptive-control-variable` `<str>`

Adaptive scale control variable: concurrency, prefill_concurrency, request_rate, or users.

#### `--adaptive-control-min` `<float>`

Minimum adaptive scale control value.
<br/>_Constraints: > 0_

#### `--adaptive-control-max` `<float>`

Maximum adaptive scale control value. Inferred from the phase target when omitted.
<br/>_Constraints: > 0_

#### `--adaptive-scale-sla` `<list>`

SLA filter for adaptive scale. Format: 'metric_tag:stat:op:threshold'. Latency-family metrics (request_latency, time_to_first_token/ttft, inter_token_latency/itl/tpot) support percentile stats; window scalar/rate metrics (request_throughput, output_token_throughput, goodput, goodput_ratio, success_rate, error_rate, cancellation_rate) support {avg, min, max}. Full metric/stat table: [Adaptive SLA metric support](tutorials/yaml-config.md#adaptive-sla-metric-support). op in {lt, le, gt, ge}; threshold is a float. Repeatable. Example: --adaptive-scale-sla 'request_latency:p95:le:30000'.

### Warmup

#### `--warmup-request-count`, `--num-warmup-requests` `<int>`
Expand Down Expand Up @@ -2412,43 +2375,6 @@ Duration in seconds to ramp prefill concurrency from 1 to target.
Duration in seconds to ramp request rate from a proportional minimum to target. Start rate is calculated as target * (update_interval / duration), ensuring correct behavior for target rates below 1 QPS. Useful for gradual warm-up of the target system.
<br/>_Constraints: > 0_

#### `--adaptive-scale`

Enable stable single-run adaptive scale control. Use --adaptive-scale-control variable:min,max:type to choose the controlled variable and bounds, and --adaptive-scale-sla metric:stat:op:threshold to define pass/fail criteria. Also requires --benchmark-duration and --adaptive-sustain-duration.
<br/>_Flag (no value required)_

#### `--adaptive-sustain-duration` `<float>`

Duration in seconds to sustain load near the discovered adaptive scale boundary.
<br/>_Constraints: > 0_

#### `--adaptive-assessment-period`, `--adaptive-scale-assessment-period` `<float>`

Duration in seconds for each adaptive scale SLA assessment window.
<br/>_Constraints: ≥ 1.0_

#### `--adaptive-scale-control` `<str>`

Compact adaptive scale control spec in variable:min,max:type form. The variable is one of concurrency, prefill_concurrency, request_rate, or users; min and max are required explicit bounds; type is int for discrete controls and float for rate controls. Examples: concurrency:1,1000:int, prefill_concurrency:1,8:int, request_rate:1,200:float, users:10,500:int. Do not combine this with expanded --adaptive-control-* flags.

#### `--adaptive-control-variable` `<str>`

Adaptive scale control variable: concurrency, prefill_concurrency, request_rate, or users.

#### `--adaptive-control-min` `<float>`

Minimum adaptive scale control value.
<br/>_Constraints: > 0_

#### `--adaptive-control-max` `<float>`

Maximum adaptive scale control value. Inferred from the phase target when omitted.
<br/>_Constraints: > 0_

#### `--adaptive-scale-sla` `<list>`

SLA filter for adaptive scale. Format: 'metric_tag:stat:op:threshold'. Latency-family metrics (request_latency, time_to_first_token/ttft, inter_token_latency/itl/tpot) support percentile stats; window scalar/rate metrics (request_throughput, output_token_throughput, goodput, goodput_ratio, success_rate, error_rate, cancellation_rate) support {avg, min, max}. Full metric/stat table: [Adaptive SLA metric support](tutorials/yaml-config.md#adaptive-sla-metric-support). op in {lt, le, gt, ge}; threshold is a float. Repeatable. Example: --adaptive-scale-sla 'request_latency:p95:le:30000'.

### Warmup

#### `--warmup-request-count`, `--num-warmup-requests` `<int>`
Expand Down
36 changes: 12 additions & 24 deletions docs/tutorials/adaptive-scale.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Use adaptive scale when you want one benchmark invocation to push a service unti

## YAML example

YAML is the preferred way to configure adaptive scale because it keeps the control variable, assessment windows, sustain period, and SLA filters together. The canonical shape uses a nested `adaptive_scale.control` block:
Adaptive scale is configured in YAML because it keeps the control variable, assessment windows, sustain period, and SLA filters together with the phase they control. The examples below set `kind: profiling` explicitly to show the current phase model, but existing configs that use canonical `warmup`/`profiling` names without `kind` remain valid because the loader infers it. The canonical shape uses a nested `adaptive_scale.control` block:

```yaml
schemaVersion: "2.0"
Expand All @@ -28,6 +28,7 @@ benchmark:
prompts: {isl: 512, osl: 128}
phases:
- name: profiling
kind: profiling
type: concurrency
concurrency: 200
prefill_concurrency: 64
Expand Down Expand Up @@ -82,6 +83,7 @@ benchmark:
osl: 16
phases:
- name: profiling
kind: profiling
type: concurrency
concurrency: 4
duration: 45
Expand Down Expand Up @@ -155,39 +157,25 @@ Exact numbers depend on your server and hardware. If the run passes at `max`, lo

For `users`, adaptive scale changes population pressure rather than acting as another spelling of request rate.

## CLI quick start
## YAML-only configuration

The CLI is useful for scripts and simple one-phase runs. Prefer YAML for reusable benchmark definitions.

```bash
aiperf profile \
--url http://localhost:8000/v1/chat/completions \
--model meta-llama/Llama-3.1-8B-Instruct \
--endpoint-type chat \
--streaming \
--concurrency 400 \
--benchmark-duration 3600 \
--adaptive-scale \
--adaptive-scale-control concurrency:1,400:int \
--adaptive-scale-assessment-period 60 \
--adaptive-sustain-duration 1800 \
--adaptive-scale-sla request_latency:p95:le:30000
```

The compact control form is `--adaptive-scale-control variable:min,max:type`. Use `int` for `concurrency`, `prefill_concurrency`, and `users`; use `float` for `request_rate`. Do not mix compact control with expanded `--adaptive-control-*` flags.
Adaptive scale does not expose standalone CLI flags. Put the adaptive settings in the target phase's `adaptive_scale` block and run the benchmark with `aiperf profile --config <file>`. General CLI flags such as `--tokenizer`, `--ui`, and `--output-artifact-dir` can still be used around the YAML benchmark definition.

## Artifacts

Adaptive scale writes these timing-owned artifacts into the run artifact directory:
Adaptive scale writes phase-scoped timing artifacts plus a manifest into the run artifact directory:

```text
adaptive_scale_events.jsonl
adaptive_scale_summary.json
phases/<phase_name>/adaptive_scale_events.jsonl
phases/<phase_name>/adaptive_scale_summary.json
adaptive_scale_manifest.json
```

A run with multiple adaptive profiling phases gets one event/summary pair per phase and one manifest entry per adaptive phase.

`adaptive_scale_events.jsonl` is an event stream for orchestration and post-processing. Each line includes `schema_version`, timestamps, `event`, `control_variable`, `control_value_before`, `control_value_after`, `boundary_value`, `last_passing_value`, `first_failing_value`, `sla_values`, and `binding_sla` fields. Pollers should key off explicit events such as `sustain_started` rather than sleeping for a fixed amount of time.

`adaptive_scale_summary.json` is the final controller summary. It records the discovered boundary, final control value, last passing value, first failing value, sustain status, throughput, sample counts, error counts, cancellation counts, and the evaluated candidate windows.
`adaptive_scale_summary.json` is the final controller summary for one adaptive phase. It records the discovered boundary, final control value, last passing value, first failing value, sustain status, throughput, sample counts, error counts, cancellation counts, and the evaluated candidate windows.

Artifact fields are intended for orchestration-facing consumers, so treat schema changes and field renames as compatibility events.

Expand Down
60 changes: 47 additions & 13 deletions docs/tutorials/yaml-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ benchmark:
entries: 500
prompts: {isl: 512, osl: 128}
phases:
- {name: warmup, type: concurrency, concurrency: 8, requests: 50, exclude_from_results: true}
- {name: profiling, type: concurrency, requests: 500}
- {name: warmup, kind: warmup, type: concurrency, concurrency: 8, requests: 50}
- {name: profiling, kind: profiling, type: concurrency, requests: 500}
artifacts:
dir: ./artifacts/my-test

Expand Down Expand Up @@ -156,11 +156,39 @@ benchmark:
datasets:
- {name: main, type: synthetic, prompts: {isl: 512, osl: 128}}
phases:
- {name: warmup, type: concurrency, concurrency: 4, requests: 50, exclude_from_results: true}
- {name: profiling, type: poisson, rate: 30.0, duration: 120}
- {name: warmup, kind: warmup, type: concurrency, concurrency: 4, requests: 50}
- {name: profiling, kind: profiling, type: poisson, rate: 30.0, duration: 120}
```

You can mix and match — the loader auto-expands `model:` into a one-element `models:` list, `dataset:` into a one-entry `datasets:` list named `default`, and a flat `phases:` block into a one-element list named `profiling`. The normalized `datasets:` form is future-facing but currently accepts exactly one dataset; multiple datasets are a roadmap item.
You can mix and match — the loader auto-expands `model:` into a one-element `models:` list, `dataset:` into a one-entry `datasets:` list named `default`, and a flat `phases:` block into a one-element profiling phase named `profiling`. Named phases are additive: existing configs that use canonical phase names such as `warmup` and `profiling` without an explicit `kind` remain valid, and the loader infers the matching semantic kind. Explicit phase lists may set `name` as the unique workflow identifier and `kind` as the semantic (`warmup` or `profiling`); custom names such as `storm_1` must set `kind: profiling` or `kind: warmup`. The normalized `datasets:` form is future-facing but currently accepts exactly one dataset; multiple datasets are a roadmap item.

Named phases are useful for long soaks with multiple profiling windows:

```yaml
benchmark:
phases:
- name: warmup
kind: warmup
type: concurrency
duration: 5m
- name: low_cancel_1
kind: profiling
type: concurrency
duration: 30m
cancellation: {rate: 5, delay: 0}
- name: storm_1
kind: profiling
type: concurrency
duration: 5m
cancellation: {rate: 50, delay: 0}
- name: recovery_1
kind: profiling
type: concurrency
duration: 30m
cancellation: {rate: 0, delay: 0}
```

Phase names must be strict identifiers (`^[A-Za-z_][A-Za-z0-9_-]*$`) and are unique case-insensitively because they are used in sweep paths and artifact directories.

### Inline datasets

Expand Down Expand Up @@ -336,6 +364,7 @@ benchmark:
prompts: {isl: 512, osl: 128}
phases:
- name: profiling
kind: profiling
type: concurrency
concurrency: 200
prefill_concurrency: 64
Expand Down Expand Up @@ -365,7 +394,7 @@ benchmark:

Adaptive scale rejects fixed ramps on the same variable it controls. For example, do not combine `control.variable: prefill_concurrency` with `prefill_ramp`. Fixed ramps for other variables are allowed.

The CLI exposes a compact sweep-like control flag for the common single-phase case: `--adaptive-scale-control variable:min,max:type`, plus repeated `--adaptive-scale-sla metric:stat:op:threshold` flags. For example: `--adaptive-scale-control "concurrency:1,1000:int" --adaptive-scale-sla "request_latency:p95:le:30000"`. Expanded `--adaptive-control-variable`, `--adaptive-control-min`, and `--adaptive-control-max` flags remain supported for advanced scripting; if expanded `--adaptive-control-max` is omitted, AIPerf infers it from the matching phase target such as `--concurrency`, `--prefill-concurrency`, `--request-rate`, or `--num-users`. Do not mix compact and expanded control forms.
Adaptive scale is a YAML-only phase feature. Configure the control variable, min/max bounds, assessment windows, sustain duration, strategy, and SLA filters in the phase `adaptive_scale` and `sla` blocks, then run the benchmark with `aiperf profile --config <file>`. Existing command-line workflows for other settings and canonical YAML phase shapes remain valid.

Adaptive scale combines SLA filters with simple AND semantics. A window passes only when every configured filter passes. Step sizing uses the smallest normalized passing margin, so the closest SLA boundary controls the next increase. There are no weights, formulas, or multi-objective scoring in single-run adaptive scale.

Expand Down Expand Up @@ -466,13 +495,16 @@ benchmark:
le: 0.10
```

Adaptive scale writes two timing-owned artifacts into the run directory:
Adaptive scale writes phase-scoped timing artifacts plus a manifest into the run directory:

```text
adaptive_scale_events.jsonl
adaptive_scale_summary.json
phases/<phase_name>/adaptive_scale_events.jsonl
phases/<phase_name>/adaptive_scale_summary.json
adaptive_scale_manifest.json
```

A run with multiple adaptive profiling phases gets one event/summary pair per phase and one manifest entry per adaptive phase.

These artifacts use schema version 2 and generic control fields such as `control_variable`, `control_value_before`, `control_value_after`, `boundary_value`, `last_passing_value`, and `first_failing_value`. Every `adaptive_window` event includes all evaluated SLA values and the binding constraint. Dynamo-style pollers should gate fault injection on explicit events such as `sustain_started` rather than fixed sleeps.

Use adaptive scale when you want continuous pressure inside one benchmark invocation. Use `sweep` or `adaptive_search` when you want offline multi-run exploration across many independent trials.
Expand Down Expand Up @@ -507,12 +539,14 @@ benchmark:
duration: 120
```

The `parameters:` keys are dot-paths into the `benchmark:` body. For lists, the second segment is the entry's `name`:
The `parameters:` keys are dot-paths into the `benchmark:` body. For phase lists, the second segment resolves in this order: numeric index, exact unique phase `name`, then legacy `phases.profiling.*` shorthand for the unique profiling-kind phase when no phase is named `profiling`:

- `phases.profiling.rate` → the phase named `profiling`, field `rate`
- `phases.storm_1.cancellation.rate` → the phase named `storm_1`, field `cancellation.rate`
- `phases.1.concurrency` → the second phase, field `concurrency`
- `phases.profiling.rate` → the phase named `profiling`, or the unique profiling-kind phase when unambiguous
- `datasets.default.prompts.isl` → the dataset named `default` (the singular `dataset:` shorthand auto-names it `default`)

The 12 most-swept phase fields also have bare-name sugar: `concurrency`, `prefill_concurrency`, `rate`, `requests`, `duration`, `sessions`, `users`, `smoothness`, `grace_period`, `concurrency_ramp`, `prefill_ramp`, `rate_ramp`. Each expands to `phases.profiling.<name>` (resolves to the unique non-warmup phase). The two forms are equivalent — see [Bare-Name Aliases](sweeps.md#bare-name-aliases-for-common-phase-fields).
The 12 most-swept phase fields also have bare-name sugar: `concurrency`, `prefill_concurrency`, `rate`, `requests`, `duration`, `sessions`, `users`, `smoothness`, `grace_period`, `concurrency_ramp`, `prefill_ramp`, `rate_ramp`. Each expands to `phases.profiling.<name>` and must still resolve unambiguously. The two forms are equivalent — see [Bare-Name Aliases](sweeps.md#bare-name-aliases-for-common-phase-fields).

Other sweep modes available in YAML:

Expand Down Expand Up @@ -578,7 +612,7 @@ aiperf profile --config benchmark.yaml \
--artifact-dir ./run-2026-05-09
```

This loads `benchmark.yaml` as the base, then overrides the *profiling* phase's `concurrency` with `32` and the artifact directory with the new path. (CLI loadgen flags overlay onto the phase named `profiling` — they don't broadcast to every named phase, so multi-phase configs need YAML edits to tweak warmup or other phases.) Useful when most of your config is stable but you want to tweak one knob from a script or CI job.
This loads `benchmark.yaml` as the base, then overrides the unique profiling phase's `concurrency` with `32` and the artifact directory with the new path. CLI loadgen flags target `kind: profiling` and work only when that target is unambiguous. If a config has multiple profiling phases, set per-phase values in YAML. Useful when most of your config is stable but you want to tweak one knob from a script or CI job.

The precedence order, lowest to highest:

Expand Down
9 changes: 9 additions & 0 deletions src/aiperf/common/accumulator_protocols.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,15 @@ class ExportContext:
phase: CreditPhase | None = None
"""Credit phase represented by this export, or None when phase-agnostic."""

phase_index: int | None = None
"""Concrete runtime phase index for phase-local exports, when available."""

phase_name: str | None = None
"""User-provided phase name for diagnostics/export metadata."""

phase_kind: str | None = None
"""Semantic phase kind for diagnostics/export metadata."""

error_summary: list[ErrorDetailsCount] | None = None
"""De-duplicated profile-run error counts to surface in the export, if any."""

Expand Down
Loading
Loading