Skip to content

BENCHMARKS.md

Latest commit

 

History

History
88 lines (66 loc) · 3.51 KB

BENCHMARKS.md

File metadata and controls

88 lines (66 loc) · 3.51 KB
Raw

Native Rust Benchmarks

Lynx performance traffic is generated by tests/benchmark_harness.rs. The harness replaces external load generators, Lua request scripts, and shell-based output parsing with a type-checked Rust workload model.

Measurement Model

A benchmark stage has a validated non-zero worker count and duration. Each Tokio worker repeatedly issues a bounded reqwest request until a common deadline. Workers self-terminate at that deadline; a second bounded join aborts stragglers so a pathological connection cannot hang a CI job.

The hot collection path uses relaxed atomic counters. Latency is sampled once per 256 requests into a short mutex-protected vector, so percentiles remain useful without making the collector the bottleneck. The report includes:

  • total, successful, and failed requests;
  • requests per second;
  • p50, p95, and p99 sampled latency;
  • client, server, unexpected-status, and transport-error counts.

These are controlled load measurements, not production throughput guarantees. Compare runs only on equivalent runner, database, network, and configuration.

Standard Suite

The standard suite creates a typed benchmark fixture and measures:

  1. cached hot redirects at 1,000, 5,000, and 10,000 workers (bounded by the configured maximum);
  2. redirects distributed across 100 URLs;
  3. API create, single-URL read, URL-list, deactivation, and health endpoints;
  4. a real 80% redirect / 15% read / 5% create mixed workload.

The mixed workload replaces the former skipped cross-port scenario because the Rust client can safely select either typed endpoint per request.

Analytics Suite

Set BENCHMARK_SUITE=analytics to run hot, 100/500-way distributed, hotspot, power-law, sustained, and analytics-query traffic. The CI workflow executes it once with analytics disabled and once enabled, then generates a typed baseline comparison in the enabled report.

Local Invocation

Start a test-only service with AUTH_MODE=none, then run:

BENCHMARK_SUITE=standard \
BENCHMARK_LABEL=local \
BENCHMARK_OUTPUT_DIR=benchmark-results \
BENCHMARK_DURATION_SECS=30 \
BENCHMARK_MAX_CONCURRENCY=10000 \
cargo test --profile profiling --locked --test benchmark_harness \
   native_external_benchmark -- --ignored --nocapture

All variables are parsed and validated by Rust:

Variable Default Meaning
BENCHMARK_SUITE standard standard or analytics
BENCHMARK_LABEL default ASCII artifact label
BENCHMARK_OUTPUT_DIR benchmark-results Output directory
BENCHMARK_DURATION_SECS 30 Duration of each stage; non-zero
BENCHMARK_MAX_CONCURRENCY 10000 Worker cap; non-zero
BENCHMARK_COMPARE_BASELINE none JSON report to compare against
LYNX_E2E_API_URL http://127.0.0.1:8080 API origin
LYNX_E2E_REDIRECT_URL http://127.0.0.1:3000 Redirect origin

Artifacts

Each run writes:

  • native-benchmark-<label>.json: typed, versioned raw metrics;
  • native-benchmark-<label>.md: stage table and optional baseline comparison;
  • INDEX.md: links to the report and JSON.

The GitHub workflow uploads these files and separately retains the existing Rust pprof flamegraphs for representative in-process redirect and API workloads. The benchmark workflow uses PostgreSQL 18, host networking, and the production-like cache/actor configuration; image and source are both pinned to the triggering commit.

See Benchmark Results for interpretation and tests/README.md for harness usage.