Skip to content

Latest commit

 

History

History
654 lines (546 loc) · 29.1 KB

File metadata and controls

654 lines (546 loc) · 29.1 KB

Configuration Guide

Table of Contents

  1. Overview
  2. Configuration Structure
  3. Full Configuration Examples
  4. Advanced Use Cases

Overview

This document provides complete documentation for all configuration options available in the Kubernetes Inference Performance Benchmark tool.

Configuration Structure

API Configuration

Controls the API interaction behavior. If SLO headers are present, each request is evaluated for SLO compliance and SLO-related metrics are reported:

api:
  type: completion             # API type (completion|chat|anthropic_messages)
  streaming: true             # Enable streaming for TTFT, ITL, and TPOT metrics
  headers:                     # Optional custom HTTP headers
    x-inference-model: llama
    x-routing-strategy: round-robin
    x-slo-tpot-ms: "2"
    x-slo-ttft-ms: "1000"
  slo_unit: "ms"               # Optional SLO unit (e.g., ms, s), default is ms
  slo_tpot_header: "x-slo-tpot-ms"        # Optional header name for TPOT SLO Header, default is x-slo-tpot-ms
  slo_ttft_header: "x-slo-ttft-ms"        # Optional header name for TTFT SLO Header, default is x-slo-ttft-ms

Data Generation

Configures the test data generation methodology:

data:
  type: mock|shareGPT|synthetic|random|shared_prefix|cnn_dailymail|billsum_conversations|infinity_instruct|otel_trace_replay|visionarena # Data generation type
  path: ./data/shareGPT/ShareGPT_V3_unfiltered_cleaned_split.json # For shareGPT type, path where dataset to be used is present. Path needs to be set for cnn_dailymail, billsum_conversations and infinity_instruct as well
  input_distribution:                                 # For synthetic/random types
    min: 10                                           # Minimum prompt length (tokens)
    max: 100                                          # Maximum prompt length
    mean: 50                                          # Average length
    std_dev: 10                                       # Standard deviation
    total_count: 100                                  # Total prompts to generate
  output_distribution:                                # Same structure as input_distribution
    min: 10
    max: 100
    mean: 50
    std_dev: 10
    total_count: 100
  shared_prefix:              # For shared_prefix type
    num_groups: 10            # Number of shared prefix groups
    num_prompts_per_group: 10 # Unique questions per group
    system_prompt_len: 100    # Shared prefix length (tokens)
    question_len: 50          # Default question length (tokens), used when question_distribution is absent
    output_len: 50            # Default output length (tokens), used when output_distribution is absent
    question_distribution:    # Optional: distribution for question lengths (overrides question_len)
      min: 10
      max: 1024
      mean: 50
      std_dev: 5
    output_distribution:      # Optional: distribution for output lengths (overrides output_len)
      min: 10
      max: 1024
      mean: 50
      std_dev: 5

Note: For otel_trace_replay type, see the OpenTelemetry Trace Replay section for complete configuration details.

Multimodal Data Generation

For VLMs, the synthetic and shared_prefix data types accept an optional multimodal block that produces images, videos, and/or audio alongside the text prompt:

data:
  type: synthetic
  input_distribution: { ... }
  output_distribution: { ... }
  multimodal:
    image:
      count: { type: uniform, min: 1, max: 2, mean: 1.5 }  # items per request
      insertion_point: 0.0                                  # 0=prefix, 1=suffix, or a Distribution
      resolutions:                                          # AnyResolution or list of weighted resolutions
        - resolution: "1080p"
          weight: 0.8
        - resolution: "4k"
          weight: 0.2
    video:
      count: { type: fixed, min: 1, max: 1, mean: 1 }
      insertion_point: 1.0
      profiles:                                             # (resolution, frames) tuples
        - profile: { resolution: "720p", frames: 32 }
          weight: 1.0
    audio:
      count: { type: fixed, min: 1, max: 1, mean: 1 }
      insertion_point: 0.5
      durations:
        - duration: 5.0
          weight: 1.0

The reportgen output adds throughput.{images,videos,audios}_per_sec, request_size_bytes, and per-modality distribution blocks (image.{count,pixels,bytes,aspect_ratio}, video.{count,frames,pixels,bytes,aspect_ratio}, audio.{count,seconds,bytes}) to summary_lifecycle_metrics.json.

Wire formats
  • Images are PNG by default. Set image.representation: jpeg for JPEG-encoded payloads (smaller, lossy) — useful when the target VLM expects JPEG or when you want wire-size closer to real client traffic.
  • Videos carry one video.representation value:
    • mp4 (default): one video_url block carrying an MP4 blob.
    • png_frames: emit frames × PNG image_url blocks at one insertion point. No server-side MP4 decode dependency; useful for prefix-cache benchmarks.
    • jpeg_frames: same as png_frames but with JPEG-encoded frames — smaller wire payload, matches client pipelines that pre-extract and JPEG-compress frames before sending to the model server.
  • Audio is 16-bit mono WAV at 16 kHz (not configurable today).
Picking values

Multimodal config is passed to the model server as-is with no model-aware validation, so the right starting point is the model's spec sheet.

  • Per-request media count — align with vLLM's --limit-mm-per-prompt (e.g. image=4,video=2,audio=2). Sending more items than the server allows fails at the wire and shows up in the lifecycle report's failures count.
  • Image resolutions — stay within the vision encoder's pixel cap (Qwen2-VL min_pixels/max_pixels, LLaVA fixed 336/672, Pixtral tile caps, etc.). Above-cap images get downsized server-side, which means reported pixels/bytes reflect what was sent, not what the model processed.
  • Video frames — most VLMs sample to a fixed num_frames budget (often 8/16/32). Sending more frames than the server samples wastes wire bytes without changing the workload the model sees. If the server doesn't accept video_url at all, switch to representation: frames.
  • Audio durations — most audio-capable models cap clips around 30 s (Qwen2-Audio, Whisper-style chunking). Longer clips fail or get truncated.
  • Effective context length — images and audio consume context tokens. Large media plus a long text prompt can exceed --max-model-len and fail.

When in doubt: start small, watch the failures count in the report, and ramp up resolutions/counts/durations once the success path is solid. There is no portable API across vLLM / SGLang / TGI to query these limits at runtime; a per-model capability registry that validates the multimodal config at load time is tracked as future work.

Load Configuration

Defines the benchmarking load pattern:

load:
  type: constant|poisson|concurrent|trace_session_replay # Load pattern type
  interval: 1.0                     # Seconds between request batches
  stages:                           # Load progression stages
    - rate: 1                       # Requests per second (CONSTANT or POISSON LOADS)
      duration: 30                  # Seconds to maintain this rate (CONSTANT or POISSON LOADS)
      concurrency_level: 3          # Level of concurrency/number of worker threads (CONCURRENT LOADS)
      num_requests: 40              # Number of requests to be processed by concurrency_level worker threads (CONCURRENT LOADS)
  num_workers: 4                    # Concurrent worker threads (default: CPU_cores)
  worker_max_concurrency: 10        # Max concurrent requests per worker
  worker_max_tcp_connections: 2500  # Max TCP connections per worker
  request_timeout: 900              # Optional: per-request timeout in seconds (applies per attempt)
  request_retries: 0                # Optional: extra attempts for faults before response headers (default: 0, no retry)
  request_retry_backoff_sec: 0.5    # Optional: base backoff before the first retry, doubled + jittered per attempt
  base_seed: 12345                  # Optional: base random seed for reproducibility (default: current time in ms)
  lora_traffic_split:               # Optional: MultiLoRA traffic splitting
    - name: adapter_1               # LoRA adapter name
      split: 0.5                    # Traffic weight (must sum to 1.0)
    - name: adapter_2
      split: 0.5

Note: trace_session_replay load type has different stage parameters. See OpenTelemetry Trace Replay for configuration details.

Retrying Transport Faults

request_retries re-sends a request that fails before response headers were obtained — a refused connection, a reset, or a connection dropped from the pool. It defaults to 0, preserving the historical behavior of failing on the first fault.

What is retried:

Fault Retried Why
Connection refused, reset, dropped from pool Yes Nothing was measured on the client side; these recover
Anything after a response was established No Re-sending could mix or discard token-level measurements and duplicate server work
Timeout No A real measurement of a slow server, and request_timeout applies per attempt
Rejected certificate, fingerprint mismatch No Configuration errors — every attempt fails identically

"No response headers" is not "the server did no work": the request bytes may already have been sent. A retry is therefore bounded and backed off rather than unconditional.

Timing. start_time stays at the logical request's dispatch and is never re-stamped per attempt, so end-to-end latency counts the failed attempt and its backoff — the workload really did wait — and schedule_delay, send_duration and achieved_rate stay correct. The cost is reported beside those numbers instead: a retried request carries info.retry_wasted_sec plus a matching OTel span attribute, and the retries block totals it across the run. A request that never succeeded counts its whole life as waste, since no attempt answered.

Retry fields are omitted rather than zeroed on anything that never retried, so a run with request_retries: 0 — or with the knob on but no faults — produces the report JSON it produced before retries existed.

Costs. request_timeout applies per attempt, so worst-case wall time per request becomes (1 + request_retries) × request_timeout plus backoff; keep request_retries small when request_timeout is large. Backoff starts at request_retry_backoff_sec, then doubles and jitters so retries do not resynchronize into a burst.

See Reports for the retries block and the CLI columns.

Load Sweeps

Defines the preprocessing phase to determine load based on target service saturation.

load:
  type: constant|poisson
  interval: 15
  sweep:                        # Automatically determine saturation point of the target service and generate stages
    type: linear|geometric      # Produce a linear distribution [1.0, saturation] of rates for num_stages or geometric distribution clustered around the saturation point
    timeout: 60                 # Length of time to run load to determine saturation
    num_stages: 5               # Number of stages to generate
    stage_duration: 180         # Duration of each generated stage
    saturation_percentile: 95   # Percentile of sampled rates to select as saturation point

Model Server

Configures connection to the model serving backend:

server:
  type: vllm                                          # Currently only vLLM supported
  model_name: "HuggingFaceTB/SmolLM2-135M-Instruct"   # Required model identifier
  base_url: "http://0.0.0.0:8000"                     # Required server endpoint
  ignore_eos: true                                    # Whether to ignore End-of-Sequence tokens
  api_key: ""                                         # Optional API key for authenticated endpoints

Metrics Collection

Sets up performance metrics collection:

metrics:
  type: prometheus|default        # Metrics backend type
  prometheus:                     # Required when type=prometheus
    url: "http://localhost:9090"  # Prometheus server URL
    scrape_interval: 15           # Metrics scrape interval (seconds)
    google_managed: false         # Whether using Google Managed Prometheus (see 'Google Managed Prometheus (GMP) Requirements' section)
    filters: []                   # List of metric names to collect

Google Managed Prometheus (GMP) Requirements

When setting google_managed: true, inference-perf queries the GMP API directly. You must configure Application Default Credentials (ADC) in your environment with sufficient permissions.

  1. Required Permissions The identity used by ADC must have the Monitoring Viewer role:

    • roles/monitoring.viewer
  2. Environment Configuration

    • GKE Cluster: Ensure the Pod is running with Workload Identity enabled and linked to a Google Service Account (GSA) with the required role.
    • GCE VM: Ensure the VM's attached Service Account has the required role.
    • Local Development: Authenticate using your user credentials:
      gcloud auth application-default login

      Note: Your personal user account must have the monitoring.viewer role on the target GCP project.

Common Error: Failing to configure these permissions will result in API errors similar to:

ERROR - error executing query: 403 Client Error: Forbidden for url: [https://monitoring.googleapis.com/v1/projects/](https://monitoring.googleapis.com/v1/projects/)...

Reporting

Controls benchmark report generation:

report:
  request_lifecycle:
    summary: true             # Generate high-level summary
    per_stage: true           # Include breakdown by load stage
    per_request: false        # Enable detailed per-request logs (verbose)
    per_request_fields:       # Control fields included in per_request_lifecycle_metrics.json
      request: true           # Include raw request payloads
      response: true          # Include raw response payloads
      info: true              # Include structured request/response metadata
      response_chunks: true   # Include raw streaming chunks inside info.response_metrics
      computed_metrics: false # Include computed per-request latency metrics (TTFT/TPOT/ITL/...)
    per_adapter: false        # Generate metrics grouped by LoRA adapter
    per_adapter_stage: false  # Generate metrics grouped by adapter and stage
    percentiles: [0.1, 1, 5, 10, 25, 50, 75, 90, 95, 99, 99.9] # List of percentiles to calculate
    use_server_output_tokens: false # Treat the server's usage.completion_tokens as the source of truth for output tokens.
    max_error_messages: 100   # Max number of unique error messages retained per error label (failures.by_label) and per bad tool call substitution entry
    use_server_output_tokens: false # Treat the server's usage.completion_tokens as the source of truth for output tokens.
  prometheus:
    summary: true             # Include Prometheus metrics summary
    per_stage: false          # Disable Prometheus stage breakdown

For metrics-only per-request reports, disable the raw fields and enable computed_metrics instead — this removes the multi-GB raw payloads while keeping full per-request latency analysis:

report:
  request_lifecycle:
    per_request: true
    per_request_fields:
      request: false
      response: false
      info: false
      response_chunks: false
      computed_metrics: true

Setting info: false removes the entire info block, including response_chunks; in that case response_chunks has no effect.

computed_metrics (off by default) adds a computed_metrics block to each successful entry with request_latency, normalized_time_per_output_token, time_to_first_token, time_per_output_token, inter_token_latency (mean), inter_token_latencies (per-token deltas), input_tokens, output_tokens, and the request's ttft_slo_sec/tpot_slo_sec SLOs — the same values (and the same tokenizer-based correction from raw streaming chunks) used to compute the summary_lifecycle_metrics.json / stage_*_lifecycle_metrics.json aggregates. Non-streamed or non-streamable requests report null for time_to_first_token, time_per_output_token, and inter_token_latency.

Storage

Configures storage for benchmark results:

storage:
  local_storage:
    path: "reports-{timestamp}"       # Local directory path
    report_file_prefix: null          # Optional filename prefix
  google_cloud_storage:               # Optional GCS configuration
    bucket_name: "your-bucket-name"   # Required GCS bucket
    path: "reports-{timestamp}"       # Optional path prefix
    report_file_prefix: null          # Optional filename prefix
  simple_storage_service:
    bucket_name: "your-bucket-name"   # Required S3 bucket
    path: "reports-{timestamp}"       # Optional path prefix
    report_file_prefix: null          # Optional filename prefix
    endpoint_url: null                # Optional custom endpoint (e.g. for S3-compatible stores)
    region_name: null                 # Optional AWS region name
    addressing_style: null            # Optional: "auto" (default), "virtual", or "path".

Tokenizer

Optional tokenizer configuration for specialized tokenization:

tokenizer:
  pretrained_model_name_or_path: "model-id"   # Required model path
  trust_remote_code: true                     # Whether to trust custom tokenizer code
  token: ""                                   # HuggingFace access token for private models
  load_timeout: 300.0                         # Deadline in seconds for loading the tokenizer,
                                              # including any download from Hugging Face Hub.
                                              # Set to null to disable. Default: 300.

load_timeout: null can only be set in a YAML config file; the --tokenizer.load_timeout CLI flag parses a float and rejects null. The deadline applies to each tokenizer construction independently; a run constructs a tokenizer in several stages (data generation, the model server client, report generation), so the worst-case total wait is a small multiple of load_timeout.

Full Configuration Examples

Minimal Configuration

data:
  type: shareGPT
load:
  type: constant
  stages:
  - rate: 1
    duration: 30
api: 
  type: chat
server:
  type: vllm
  model_name: HuggingFaceTB/SmolLM2-135M-Instruct
  base_url: http://0.0.0.0:8000

Advanced Configuration

load:
  type: constant
  stages:
  - rate: 1
    duration: 30
api: 
  type: completion
server:
  type: vllm
  model_name: HuggingFaceTB/SmolLM2-135M-Instruct
  base_url: http://0.0.0.0:8000
  ignore_eos: true
tokenizer:
  pretrained_model_name_or_path: HuggingFaceTB/SmolLM2-135M-Instruct
data:
  type: random
  use_chat_template: false  # wrap each prompt in the tokenizer's chat template (single user turn) before sending.
                            # Input lengths then target the fully templated prompt, so the server-side prefill
                            # token count still matches the configured length. Requires a tokenizer with a chat
                            # template, and every input length must exceed the template's fixed token overhead.
  input_distribution:
    min: 10             # min length of the synthetic prompts
    max: 100            # max length of the synthetic prompts
    mean: 50            # mean length of the synthetic prompts
    std_dev: 10         # standard deviation of the length of the synthetic prompts
    total_count: 100    # total number of prompts to generate to fit the above mentioned distribution constraints
  output_distribution:
    min: 10             # min length of the output to be generated
    max: 100            # max length of the output to be generated
    mean: 50            # mean length of the output to be generated
    std_dev: 10         # standard deviation of the length of the output to be generated
    total_count: 100    # total number of output lengths to generate to fit the above mentioned distribution constraints
metrics:
  type: prometheus
  prometheus:
    url: http://localhost:9090
    scrape_interval: 15
report:
  request_lifecycle:
    summary: true
    per_stage: true
    per_request: true
  prometheus:
    summary: true
    per_stage: true

To Run Inference Perf Offline

load:
  type: constant
  stages:
  - rate: 1
    duration: 30
api:
  type: chat
server:
  type: vllm
  model_name: ./models/SmolLM2-135M-Instruct
  base_url: http://0.0.0.0:8000
  ignore_eos: true
tokenizer:
  pretrained_model_name_or_path: ./models/SmolLM2-135M-Instruct
data:
  type: shareGPT
  path: ./data/shareGPT/ShareGPT_V3_unfiltered_cleaned_split.json # path to the downloaded shareGPT dataset
metrics:
  type: prometheus
  prometheus:
    url: http://localhost:9090
    scrape_interval: 15
report:
  request_lifecycle:
    summary: true
    per_stage: true
    per_request: false
  prometheus:
    summary: true
    per_stage: true

Advanced Use Cases

OpenTelemetry Trace Replay

Replay real-world LLM workloads captured as OpenTelemetry traces. This feature enables benchmarking with production traffic patterns, including complex dependency graphs, multi-turn conversations, and agent workflows.

Overview

OTel trace replay reconstructs the original call graph from trace files, preserving:

  • Sequential dependencies — requests that must wait for predecessors
  • Parallel fan-outs — concurrent requests with no dependencies
  • Shared-prefix patterns — requests sharing common message history (KV-cache opportunities)
  • Output-aware replay — substitutes recorded assistant messages with actual generated text for realistic growing-context behavior

How It Works

  1. Trace → Replay Graph: Each trace file is converted to a directed acyclic graph (DAG) where:

    • LLM spans become nodes
    • Dependencies are inferred from message content (assistant messages matching predecessor outputs)
    • Timing gaps between calls are preserved as wait_ms delays
  2. Session-Based Execution: Each trace file represents one session. The load generator controls:

    • How many sessions run concurrently (concurrent_sessions)
    • How many sessions to process per stage (num_sessions)
    • Optional rate limiting for session starts (session_rate)
  3. Output Substitution: When a request depends on a predecessor's output, the recorded assistant message is replaced at runtime with the actual generated text, ensuring realistic KV-cache behavior for multi-turn conversations and agent chains.

Configuration

data:
  type: otel_trace_replay
  otel_trace_replay:
    # Source — specify one:
    trace_files:                                  # List of specific trace files
      - "path/to/trace1.json"
      - "path/to/trace2.json"
    trace_directory: "path/to/traces/"            # OR: all .json files in directory

    # Model configuration
    use_static_model: true                        # Override recorded model names
    static_model_name: "my-model"                 # Model to use for all requests
    model_mapping:                                # OR: remap per recorded name
      "gpt-4": "my-model"
      "gpt-3.5-turbo": "my-other-model"

    # Generation parameters
    default_max_tokens: 1000                      # Fallback if output tokens are set to 0 in the otel file

    # Error handling
    include_errors: false                         # Skip spans with error status, that is, status != 0 (default)
    skip_invalid_files: true                      # Skip unparseable trace files during replay

    # Tool-call mitigation (client-side, default disabled)
    bad_tool_call_handling: none                  # none|use_recorded — see docs/otel_trace_replay.md#bad-tool-call-handling

    # Output replay fidelity (default disabled)
    disable_output_substitution: false            # true = send recorded assistant outputs as-is (no live substitution); conflicts with inject_random_session_id / duplicate_sessions_target

load:
  type: trace_session_replay                      # Required for otel_trace_replay
  stages:
    - concurrent_sessions: 4                      # Max sessions active simultaneously
      num_sessions: 20                            # Run 20 sessions in this stage
      session_rate: 2.0                           # Optional: start max 2 sessions/sec
      timeout: 300                                # Optional: stage timeout in seconds
  num_workers: 4                                  # Worker processes
  worker_max_concurrency: 10                      # Max concurrent requests per worker

Stage Configuration

concurrent_sessions (required): Controls session-level concurrency

  • 0 = unlimited (all sessions active at once, stress test mode)
  • N > 0 = at most N sessions active; when one completes, the next starts

num_sessions (optional): Number of sessions to run in this stage

  • If omitted, runs all remaining sessions in the corpus
  • Stages advance through the corpus sequentially (like standard load stages)

session_rate (optional): Rate limit for starting new sessions

  • Omit for no rate limiting
  • Useful for controlled ramp-up scenarios

timeout (optional): Wall-clock safety limit

  • If exceeded, in-flight sessions are cancelled and stage exits as FAILED

Trace File Format

Traces must be JSON files with a spans array. Each LLM span requires:

{
  "span_id": "unique-id",
  "trace_id": "trace-id",
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-01-01T00:00:01Z",
  "name": "chat gpt-4",
  "attributes": {
    "gen_ai.request.model": "gpt-4",
    "gen_ai.input.messages": "[{\"role\":\"user\",\"content\":\"hello\"}]",
    "gen_ai.output.text": "hi there",
    "gen_ai.usage.prompt_tokens": 10,
    "gen_ai.usage.completion_tokens": 5
  }
}

Token counts are read from gen_ai.usage.prompt_tokens / gen_ai.usage.completion_tokens (also accepts input_tokens / output_tokens). If absent, a 4 chars/token estimate is used.

Example Configurations

Simple Sequential Replay:

data:
  type: otel_trace_replay
  otel_trace_replay:
    trace_files:
      - "examples/otel/test_traces/simple/simple_chain.json"
    use_static_model: true
    static_model_name: "HuggingFaceTB/SmolLM2-135M-Instruct"
    default_max_tokens: 100

load:
  type: trace_session_replay
  stages:
    - concurrent_sessions: 1  # One session at a time
  num_workers: 4
  worker_max_concurrency: 10

server:
  type: vllm
  base_url: "http://localhost:8000"
  model_name: "HuggingFaceTB/SmolLM2-135M-Instruct"

Multi-Stage with Rate Limiting:

data:
  type: otel_trace_replay
  otel_trace_replay:
    trace_directory: "examples/otel/test_traces/advanced"
    use_static_model: true
    static_model_name: "my-model"

load:
  type: trace_session_replay
  stages:
    - concurrent_sessions: 2
      num_sessions: 10
      session_rate: 1.0      # Warm-up: 2 concurrent, 1/sec start rate
    - concurrent_sessions: 5
      num_sessions: 20
      session_rate: 2.0      # Ramp-up: 5 concurrent, 2/sec start rate
    - concurrent_sessions: 10
                             # Final stage: 10 concurrent, all remaining sessions
  num_workers: 8
  worker_max_concurrency: 20

Use Cases

  • Production Traffic Replay: Benchmark with real user interaction patterns
  • Agent Workflow Testing: Replay complex multi-step agent traces with tool calls
  • Multi-Turn Conversation Analysis: Test KV-cache efficiency with realistic conversation flows
  • Dependency Graph Validation: Verify server behavior under complex request dependencies
  • Comparative Analysis: Replay the same traces against different model configurations

Architecture Notes

Unlike standard data generators that produce independent requests, OTel trace replay operates at the session granularity. Each session is a complete trace file with an internal dependency graph. The load generator:

  1. Activates sessions according to concurrent_sessions limit
  2. Dispatches all events for a session immediately (workers handle internal parallelism)
  3. Each event blocks until its predecessors complete and outputs are available
  4. Tracks session completion and starts new sessions as slots become available

This design preserves the causal structure of the original workload while allowing the load generator to control session-level concurrency and throughput.