FairServeLab

FairServeLab is a deterministic, trace-driven experiment framework for comparing admission control and multi-tenant request scheduling under identical synthetic workloads.

It replays each CSV trace with FIFO, Strict Priority, Round Robin, Deficit Round Robin (DRR), and Self-Clocked Fair Queueing (SCFQ), then writes request-level, tenant-level, queue, latency, throughput, deadline, and fairness evidence.

FairServeLab is not a production rate limiter, a replacement for Envoy, or evidence of real server throughput. Service times are known from the offline trace and all reported performance is simulated logical time.

Why FairServeLab

Production systems such as Envoy and Netflix concurrency-limits enforce live traffic policies. FairServeLab addresses a different question: when arrivals, request costs, worker count, capacity, and event semantics remain fixed, how does the selected admission or scheduling policy change latency, outcomes, and per-tenant fairness?

The framework makes that comparison inspectable through versioned trace bytes, SHA-256 provenance, immutable lifecycle records, explicit formulas, controlled scenarios, and deterministic result artifacts. See Related Work and Project Boundary for the scoped comparison.

V1 capabilities

• Immutable, validated request and terminal-result contracts
• Exact lifecycle transition rules
• UTF-8 CSV trace validation and SHA-256 provenance
• Scenario schema v2 with pinned trace SHA-256 preflight checks
• Fixed-seed general-purpose synthetic trace generator
• Fixed-capacity admission control
• Deterministic completion → timeout → arrival → dispatch event ordering
• Identical non-preemptive worker pool across policies
• FIFO, Strict Priority, Round Robin, weighted DRR, and SCFQ
• p50/p95/p99 latency and queueing delay
• Accepted, rejected, timeout, and deadline-miss outcomes
• Per-tenant throughput and slowdown
• Weight-normalized Jain fairness
• Queue-length time series and time-weighted average
• JSON, CSV, JSONL, Markdown, and PNG artifacts
• Eleven controlled synthetic benchmark scenarios
• Unit, integration, property-based, and deterministic replay tests
• Strict static type checking and a 90% test-coverage gate
• GitHub Actions CI

Verified benchmark snapshot

The built-in catalog contains 11 scenarios and 53 declared policy replays. A single command runs the full suite and writes a provenance index. The checked snapshot below comes from the weighted_saturation scenario: all five policies complete 30 requests inside the same horizon at 200,000 requests per simulated second, while their weight-normalized service allocation differs.

See Reproduced V1 Results for exact trace hashes, scenario hashes, tables, reproduction commands, and interpretation limits.

Installation

FairServeLab requires Python 3.11 or newer.

python3 -m venv .venv
.venv/bin/python -m pip install '.[test]'

Reinstall after changing package source so the environment contains the current code. Repository tests also add src explicitly and do not depend on editable install path hooks.

Quick start

Materialize the versioned built-in benchmark inputs:

.venv/bin/fairservelab materialize-suite --root benchmarks

Validate a trace or a complete scenario without starting a simulation:

.venv/bin/fairservelab validate-trace \
  --trace benchmarks/traces/weighted_saturation.csv
.venv/bin/fairservelab validate-scenario \
  --scenario benchmarks/scenarios/weighted_saturation.json

Run all 11 controlled scenarios:

.venv/bin/fairservelab benchmark-suite \
  --root benchmarks \
  --output results/v1

Run one controlled comparison:

.venv/bin/fairservelab benchmark \
  --scenario benchmarks/scenarios/weighted_saturation.json \
  --output results/weighted_saturation

Run one scheduler directly:

.venv/bin/fairservelab run \
  --trace benchmarks/traces/noisy_neighbor.csv \
  --scheduler drr \
  --workers 1 \
  --capacity 80 \
  --horizon 100 \
  --drr-quantum 10 \
  --starvation-threshold 40 \
  --output results/noisy_neighbor-drr

Generate a separate fixed-seed trace:

.venv/bin/fairservelab generate \
  --output /tmp/fairservelab-trace.csv \
  --requests 200 \
  --tenants 4 \
  --seed 1729 \
  --horizon 1000000

Verification

.venv/bin/python -m ruff check .
.venv/bin/python -m ruff format --check .
.venv/bin/python -m mypy
.venv/bin/python -m pytest
.venv/bin/python -m build

Documentation

• Architecture
• Contracts
• Event semantics
• Scheduler specifications
• Metrics
• Benchmark methodology
• Reproduced V1 results
• Related work and project boundary
• Limitations
• Benchmark catalog
• Contributing

Output contract

Each run writes:

run.json
requests.csv
tenants.csv
queue_timeseries.csv
events.jsonl
report.md
plots/queue_length.png
plots/per_tenant_service.png

A multi-policy scenario additionally writes comparison.json, comparison.csv, comparison.md, and three comparison plots.

Run and comparison JSON records include the FairServeLab software version and trace SHA-256. Scenario-driven outputs also include the exact scenario SHA-256.

A full-suite run writes suite.json and suite.md at its root. The JSON index records every scenario and trace digest, replay count, and relative result directory.

License

MIT