Model distributed-tracing sampling strategies against a traffic profile and find out what each one actually costs you — in bytes on the wire, gigabytes at rest, dollars per month, collector memory, error traces you silently drop, and the probability you ever capture the rare bug you are hunting.
It ships as two interfaces over one model:
- a client-side web app (
web/) with live sliders, hand-rolled SVG charts, shareable URLs and CSV/Markdown export — no bundler, no CDN, no network access at runtime; - a Python CLI (
python -m samplingsim) with the same engine, a TOML config format andtable/json/csv/markdown/detailoutput.
The Python engine is the source of truth. The JavaScript engine mirrors it, and both are pinned to a shared JSON fixture that both test suites assert against, so the two implementations cannot silently disagree. See How the two engines stay in sync.
If you want the conceptual background rather than the arithmetic, the site this tool accompanies has a walkthrough of choosing between head-based and tail-based sampling.
- Why this exists
- Requirements and setup
- Quick start
- The strategies
- The maths, written out
- Assumptions, stated loudly
- Worked example
- CLI reference
- Config file format
- Cost presets
- The web app
- How the two engines stay in sync
- Tests
- Project layout
- Limitations
- Further reading
- License
Sampling arguments usually stall on three unmeasured claims:
- "Tail sampling is too expensive." — Expensive where? Tail sampling does not reduce the app-to-collector bandwidth at all, because every span has to arrive before a decision can be made. It reduces what you store. Those are different budgets, and conflating them produces the wrong decision.
- "1% head sampling is fine." — Fine for dashboards, perhaps. At 1% you are throwing away 99% of your error traces, and if the bug you are chasing happens once in ten million requests you will wait weeks for a single captured instance.
- "We'll just turn on tail sampling." — The tail sampler holds every span of every
in-flight trace in memory for
decision_waitseconds, sized for peak traffic, not average. That product is usually gigabytes, and it is the number that OOMs collectors.
This tool computes all three, from the same traffic profile, for every strategy at once.
This project is not published to any package registry. Clone it and run it from source.
git clone https://github.com/distributed-tracing-request-correlation/sampling-strategy-simulator.git
cd sampling-strategy-simulatorPython CLI — Python 3.11 or newer (it uses tomllib from the standard library). The
tool itself has no third-party dependencies. Run it straight from the repository root:
python3 -m samplingsim --helpWeb app — any modern browser. Because it is built from ES modules, it must be served
over HTTP rather than opened as a file:// URL:
python3 -m http.server 8000 --directory web
# then open http://localhost:8000/Nothing is fetched at runtime and nothing is uploaded; the page works with the network disconnected.
Running the tests — the only development dependency is pytest:
python3 -m pip install -r requirements-dev.txt
python3 -m pytest -q
node --test tests/js/*.test.mjsNode 20+ is needed for the JavaScript suite. There are no npm dependencies.
Compare every strategy against the built-in default profile:
$ python3 -m samplingsim --compare --sort rare-24h
Strategy Keep Whole Traces/s Spans/s Wire Stored $/month Errors lost/day P(rare|24h) TTFC Buffer/instance
------------------------------------ ----- --------- -------- ------- ----------- --------- ------- --------------- ----------- -------- ---------------
No sampling 100% 100% 1,000 20,000 10.00 MB/s 1.51 TB $284.78 0 100% 83.3 min -
Tail: all errors + 1% baseline 2.98% 100% 29.8 596 10.00 MB/s 45.06 GB $251.04 0 100% 83.3 min 200.00 MB
Tail: errors OR slow + 1% baseline 3.95% 100% 39.5 790.04 10.00 MB/s 59.73 GB $251.37 0 100% 83.3 min 200.00 MB
Head probabilistic 10% 10% 100% 100 2,000 1.00 MB/s 151.20 GB $253.48 1,555,200 82.24% 13.9 h -
Head probabilistic 10%, inconsistent 10% 1.00e-09% 100 2,000 1.00 MB/s 151.20 GB $253.48 1,555,200 82.24% 13.9 h -
Head rate limit 5/s/service 6% 100% 60 1,200 600.00 kB/s 90.72 GB $252.09 1,624,320 64.54% 23.1 h -
Adaptive budget 50 traces/s 5% 100% 50 1,000 500.00 kB/s 75.60 GB $251.74 1,641,600 57.85% 27.8 h -
Tail: slow traces + 1% baseline 1.99% 100% 19.9 398 10.00 MB/s 30.09 GB $250.69 1,693,613 29.1% 2.9 d 200.00 MB
Head probabilistic 1% 1% 100% 10 200 100.00 kB/s 15.12 GB $250.35 1,710,720 15.87% 5.8 d -Three things fall out of that table immediately. The Wire column is identical for every
tail strategy and for no sampling at all — tail sampling never reduces ingest bandwidth.
The Whole column shows that head sampling applied independently per service across twelve
services leaves you with essentially no complete traces. And Errors lost/day is zero for
every error-aware tail policy and over 1.5 million for every head policy.
Column meanings:
| Column | Meaning |
|---|---|
Keep |
Fraction of traces retained, on average |
Whole |
Fraction of retained traces that are complete rather than fragmented |
Traces/s, Spans/s |
Retained throughput |
Wire |
Bytes/sec on the app-to-collector link, uncompressed |
Stored |
Bytes at rest in steady state, compressed, over the retention window |
$/month |
Estimated monthly cost under the selected preset |
Errors lost/day |
Error traces dropped by sampling per day |
P(rare|24h) |
Probability of capturing the rare event at least once in 24 hours |
TTFC |
Mean time to first capture of the rare event |
Buffer/instance |
Tail sampler decision-wait memory per collector instance |
| CLI name | Kind | What it models |
|---|---|---|
none |
none |
Keep everything. The baseline. |
head |
head_probabilistic |
Fixed-rate decision at the root, propagated via sampled in the trace flags — parent-based consistent sampling, so kept traces are whole. |
head-inconsistent |
head_probabilistic (consistent = false) |
Each service samples independently at the same rate. Same span volume, shredded traces. |
rate-limit |
head_rate_limit |
Each service admits at most N traces/sec. |
adaptive |
adaptive |
Head rate continuously retuned to hold a fixed trace budget. |
tail-errors |
tail_errors |
Keep every trace containing an error, plus a probabilistic baseline. |
tail-latency |
tail_latency |
Keep every trace over a latency threshold, plus a baseline. |
tail-composite |
tail_composite |
Errors OR slow OR baseline — the shape most production tail configurations actually take. |
The head-inconsistent variant is included because it is a real and common misconfiguration
rather than a hypothetical: it costs exactly as much as consistent head sampling and gives
you almost nothing usable. See
span lifecycle and parent-child relationships
for why a fragmented trace is so much less useful than a whole one.
Symbols, all taken from the profile:
| Symbol | Profile field | Meaning |
|---|---|---|
R |
requests_per_sec |
Root requests per second; one request is one trace |
S |
spans_per_trace |
Mean spans in a complete trace |
B |
span_size_bytes |
Mean uncompressed span size |
e |
error_rate |
Fraction of traces containing an error span |
s |
slow_trace_rate |
Fraction of traces over the latency threshold |
N |
rare_event_one_in |
Rare event occurs once in N requests |
D |
retention_days |
Retention window |
V |
num_services |
Services participating in a trace |
P |
peak_to_avg_ratio |
Peak rps ÷ average rps |
C |
compression_ratio |
Uncompressed ÷ compressed bytes |
W |
decision_wait_seconds |
Tail sampler decision_wait |
M |
memory_overhead_factor |
In-memory bytes per buffered span ÷ its wire size |
I |
collector_instances |
Tail-sampling collector instances |
Each strategy produces four fractions: k (average), k_peak, k_err (of error traces),
and k_rare (of rare-event traces). Writing A ∪ B = A + B − A·B for the union of two
independent events:
none keep k = 1
head probabilistic k = p k_err = p k_rare = p
head rate limit k = min(1, L·V / R) k_peak = min(1, L·V / (R·P))
adaptive k = min(1, T / R) k_peak = min(1, T / (R·P))
tail errors k = e ∪ b k_err = 1 k_rare = 1 if the rare event is an error, else b
tail latency k = s ∪ b k_err = k k_rare = k
tail composite k = (e ∪ s) ∪ b k_err = 1 k_rare = 1 if the rare event is an error, else s ∪ b
where p is the head rate, b the tail baseline rate, L the per-service rate limit and
T the adaptive trace budget. Head and adaptive strategies keep error traces at exactly the
same rate as everything else, because they cannot see whether the trace failed — that is the
whole point of the head/tail distinction.
Consistent (parent-based) head sampling propagates the root's decision, so every retained trace is whole:
complete_trace_fraction = 1
If each of the V services samples independently at rate p, the root's decision is one of
V independent draws, so among the traces whose root was sampled the fraction that are
complete is:
complete_trace_fraction = p^(V − 1)
At p = 0.1 across 12 services that is 10⁻¹¹. You pay full price for 10% of the spans and
receive approximately zero usable traces.
traces/s retained = R · k
spans/s retained = R · k · S
spans/day retained = R · k · S · 86400
This is the distinction the tool exists to make. Head sampling drops the trace inside the SDK, so the bytes never leave the process. Tail sampling must see every span before it can decide, so the app-to-collector link carries the full firehose regardless of how aggressive the policy is:
raw bytes/s = R · S · B
wire bytes/s (head) = R · k · S · B
wire bytes/s (tail) = R · S · B ← unchanged by sampling
backend bytes/s = R · k · S · B ← what reaches storage, either way
compressed = bytes/s ÷ C
peak = the same expression with R replaced by R·P (and k by k_peak)
Storage is the steady state implied by the retention window, not a cumulative total:
stored bytes/day = backend bytes/s · 86400 ÷ C
stored bytes = stored bytes/day · D
GB = bytes ÷ 10⁹
The number that OOMs people. A tail sampler holds every span of every in-flight trace until
decision_wait elapses, and it must survive the peak, not the average:
spans buffered = R · P · S · W
buffer bytes = spans buffered · B · M
buffer per instance = buffer bytes ÷ I
With 1,000 rps average, a 3× peak, 20 spans/trace, 500 B/span, a 10 s decision_wait and a
2× in-memory overhead, that is 600,000 spans and 600 MB spread across your collector fleet —
and it scales linearly with all five factors at once. Size the memory_limiter processor
above it, and remember that every span of a trace must reach the same collector instance,
so you need a trace-aware load balancer in front. See
configuring the batch and memory_limiter processors.
A month is 30.4375 days (365.25 ÷ 12) and a GB is 10⁹ bytes, which is how storage is billed.
ingest GB/month = billed bytes/s · 86400 · 30.4375 ÷ 10⁹
stored GB = stored bytes ÷ 10⁹
spans/month = spans/s retained · 86400 · 30.4375
total $/month = ingest GB/month · price_per_gb_ingest
+ stored GB · price_per_gb_stored_month
+ spans/month/10⁶ · price_per_million_spans
+ fixed_monthly
billed bytes/s is the post-sampling backend volume by default. If your vendor runs the
tail sampler on their side, you are billed on everything you send them — set
--bill-ingest-pre-sampling and the tail strategies are billed on the raw firehose instead.
error traces/day = R · 86400 · e
captured/day = error traces/day · k_err
dropped/day = error traces/day · (1 − k_err)
The headline number. A rare event occurs on any single request with probability 1/N. It is
captured only if it also survives sampling, so the per-request capture probability is:
q = (1 / N) · k_rare
Over a window of t seconds there are n = R · t requests, each an independent trial, so:
P(at least one capture in t) = 1 − (1 − q)^n
which is reported for t = 1 h, 24 h and 7 d. Captures arrive as a Poisson process of rate
R · q, so:
expected captures/day = R · 86400 · q
mean time to first capture = 1 / (R · q)
median time to first capture = ln(2) / (R · q)
When q = 0 — 0% sampling, or an errors-only tail policy against a rare event that does not
produce an error span — the capture probability is 0 and the time to capture is reported as
never (null in JSON, because JSON has no infinity).
Implementation note: 1 − (1 − q)^n is computed as −expm1(n · log1p(−q)). Evaluating it
directly loses all precision when q is around 10⁻⁸, which is exactly the regime this
tool operates in. As q → 0 it converges to the Poisson form 1 − e^(−R·q·t), and the test
suite checks that convergence.
Every one of these is a place where the model is simpler than reality. If a number from this tool is going into a budget or a design document, read this list first.
- One root request produces exactly one trace of
spans_per_tracemean spans. Mean span counts hide a long tail; a p99 trace can be an order of magnitude larger. - Every span is the same size. In reality root spans and database spans differ a lot, and a single service logging large attributes can dominate the bill.
- Head sampling is parent-based and consistent unless the strategy explicitly says otherwise.
- Tail sampling sees 100% of spans. True for a collector-based tail sampler; not true if you pre-filter with a head sampler upstream of it, which is a common and sensible hybrid this model does not represent.
- Errors and slow traces are treated as independent when policies are OR-ed. In practice errors correlate strongly with latency, so the true union is somewhat smaller than modelled — the composite policy's keep fraction is a slight overestimate.
- The rare event is an independent Bernoulli trial per request. Real incidents are bursty and correlated; a burst is far easier to catch than the same number of events spread uniformly, so the reported probabilities are conservative for bursty failures.
- Compression is a single flat ratio applied to stored bytes. Real ratios vary with attribute cardinality and batch size.
- The cost model is linear and has no tiers, commitments or negotiated discounts. The presets are estimates assembled from public list prices, not vendor quotes. Replace them with your own contract before anyone budgets against the output.
memory_overhead_factoris a rule of thumb. Actual in-process cost per buffered span depends on the collector's build and your attribute cardinality. Measure it if it matters; 2× is a reasonable starting estimate, not a guarantee.- Query load, index overhead and replication are not modelled. An Elasticsearch index
can double the on-disk footprint, and replicas multiply it; the
jaeger-elasticsearchpreset folds an approximation of this into its per-GB price.
examples/ecommerce.toml describes a mid-size checkout path: 4,200 requests/sec, 34 spans
per trace, 560 B per span, a 1.2% error rate, 21-day retention, a 5.5× Black-Friday peak,
and a payment double-charge bug that occurs roughly once in 25 million requests.
$ python3 -m samplingsim --config examples/ecommerce.toml --sort rare-24h
Strategy Keep Whole Traces/s Spans/s Wire Stored $/month Errors lost/day P(rare|24h) TTFC Buffer/instance
---------------------------------- ------ ----- -------- ------- ---------- --------- ------- --------------- ----------- ------ ---------------
No sampling 100% 100% 4,200 142,800 79.97 MB/s 18.14 TB $817.15 0 100% 1.7 h -
Tail: all errors + 1% baseline 2.188% 100% 91.9 3,124 79.97 MB/s 396.83 GB $409.13 0 100% 1.7 h 1.76 GB
Tail: errors OR slow + 1% baseline 3.166% 100% 132.98 4,521 79.97 MB/s 574.23 GB $413.21 0 100% 1.7 h 1.76 GB
Head probabilistic 5% 5% 100% 210 7,140 4.00 MB/s 906.84 GB $420.86 4,136,832 51.6% 33.1 h -
Adaptive budget 100 traces/s 2.381% 100% 100 3,400 1.90 MB/s 431.83 GB $409.93 4,250,880 29.22% 2.9 d -
Head rate limit 10/s/service 2.143% 100% 90 3,060 1.71 MB/s 388.64 GB $408.94 4,261,248 26.73% 3.2 d -Read the bottom row against the third. The head rate limiter and the composite tail policy cost $408.94 and $413.21 a month — a difference of $4.27, well inside the error bars on any of these price estimates. For that $4.27 the tail policy drops zero error traces instead of 4.26 million a day, and expects to capture the double-charge bug in 1.7 hours instead of 3.2 days.
What it costs instead is bandwidth and memory: 79.97 MB/s on the app-to-collector link rather than 1.71 MB/s, and 1.76 GB of decision buffer on each of six collectors. That is the real trade, and it is a capacity-planning problem rather than a line item.
The long-form view of a single strategy:
$ python3 -m samplingsim --config examples/ecommerce.toml --strategy tail-composite --format detail
Tail: errors OR slow + 1% baseline
==================================
OR several policies together — errors, latency, plus a baseline — which is what most production tail configurations actually look like.
Background: When to use tail-based sampling for microservices — https://www.distributed-tracing.com/distributed-tracing-fundamentals-architecture/choosing-between-head-based-and-tail-based-sampling/when-to-use-tail-based-sampling-for-microservices/
Sampling
keep fraction (avg) 3.166%
keep fraction (peak) 3.166%
error traces kept 100%
rare-event traces kept 100%
Volume
traces/s 132.98 of 4,200
spans/s 4,521 of 142,800
spans/day 390,633,353
Bandwidth (app -> collector)
uncompressed 79.97 MB/s
compressed 8:1 10.00 MB/s
at peak x5.5 439.82 MB/s
reduction vs no sampling 1.0x
Storage (compression 8:1, retention 21 d)
per day 27.34 GB
steady state 574.23 GB
Tail sampler decision buffer (decision_wait 12s, sized at peak)
spans held in memory 9,424,800
total 10.56 GB
per instance (x6) 1.76 GB
This is the number that OOMs tail-sampling collectors. Size the
memory_limiter processor above it, and remember every span of a
trace must reach the same instance.
Cost (tempo-on-s3-estimate preset — an estimate, not a quote)
ingest 6,658.3 GB/month $0.00
storage 574.2 GB retained $13.21
spans 11,889.9 M/month $0.00
fixed $400.00
total $413.21/month
What you would miss
error traces/day 4,354,560
captured 4,354,560
dropped 0
Rare event (1 in 25,000,000 requests)
occurrences/day 14.52
captured/day 14.52
P(>=1 capture) 1h 45.38%
P(>=1 capture) 24h 100%
P(>=1 capture) 7d 100%
mean time to capture 1.7 h
median 68.8 min(One line of the article link is elided above for width; the real output prints the full URL.)
Note at peak x5.5 — 439.82 MB/s. Sizing the collector ingress for the 79.97 MB/s average
is how you lose traces on the day you most want them.
python3 -m samplingsim [options]
Every flag overrides the corresponding field from --config, or the built-in default.
| Flag | Default | Meaning |
|---|---|---|
--rps |
1000 |
Root requests per second; one request is one trace |
--spans-per-trace |
20 |
Mean spans in a complete trace |
--span-size |
500 |
Mean uncompressed span size in bytes (OTLP protobuf) |
--error-rate |
0.02 |
Fraction of traces containing an error span, 0–1 |
--slow-rate |
0.01 |
Fraction of traces over the latency threshold; a p99 threshold means 0.01 |
--rare-one-in |
5000000 |
The rare event occurs once in this many requests |
--rare-event-is-error / --no-rare-event-is-error |
on | Whether the rare event surfaces as an error span |
--retention-days |
14 |
Days retained data is kept |
--services |
12 |
Services participating in a trace |
--peak-ratio |
3 |
Peak rps ÷ average rps |
--compression |
8 |
Uncompressed:compressed byte ratio |
--decision-wait |
10 |
Tail sampler decision_wait, in seconds |
--memory-overhead |
2 |
In-memory bytes per buffered span ÷ its wire size |
--collectors |
3 |
Tail-sampling collector instances |
--bill-ingest-pre-sampling / --no-... |
off | Bill ingest on bytes received before tail sampling |
| Flag | Default | Meaning |
|---|---|---|
--strategy NAME |
— | Strategy to evaluate; repeatable. One of none, head, head-inconsistent, rate-limit, adaptive, tail-errors, tail-latency, tail-composite |
--compare |
— | Evaluate the full default strategy set |
--rate |
0.1 |
Head sampling rate for head / head-inconsistent |
--limit |
5 |
Traces/sec/service for rate-limit |
--target |
50 |
Trace budget in traces/sec for adaptive |
--baseline |
0.01 |
Baseline probabilistic rate inside tail policies |
With no --strategy and no [[strategies]] in a config file, the full default set is
evaluated.
| Flag | Default | Meaning |
|---|---|---|
--cost-preset |
self-hosted-tempo-s3 |
One of self-hosted-tempo-s3, jaeger-elasticsearch, vendor-per-span, vendor-per-gb, free |
--cost-per-gb-ingest |
from preset | USD per GB accepted by the backend |
--cost-per-gb-stored-month |
from preset | USD per GB held for a month |
--cost-per-million-spans |
from preset | USD per million retained spans |
--cost-fixed-monthly |
from preset | USD per month independent of volume |
--list-presets |
— | Print the presets with their notes and exit |
Precedence is preset → [cost] table in the config file → explicit --cost-* flags.
| Flag | Default | Meaning |
|---|---|---|
--format |
table |
table, json, csv, markdown or detail |
--sort |
cost |
cost, traces, spans, storage, errors-dropped, rare-24h or none |
--config FILE |
— | TOML file with [profile], [cost] and [[strategies]] |
--explain |
— | Append a one-line explainer and article link per strategy (table / markdown) |
--format json emits the complete result object for every strategy, including fields the
table has no room for. It is the interface to use if you want to plot this yourself:
$ python3 -m samplingsim --strategy tail-composite --format json \
| python3 -c 'import json,sys; r=json.load(sys.stdin)["results"][0]; print(r["collector"])'
{'applies': True, 'spans_buffered': 600000.0, 'buffer_bytes_total': 600000000.0, 'buffer_bytes_per_instance': 200000000.0, 'buffer_gb_per_instance': 0.2}TOML, with three optional sections. Anything omitted falls back to the built-in default, and command line flags override the file.
[profile]
requests_per_sec = 4200
spans_per_trace = 34
span_size_bytes = 560
error_rate = 0.012
slow_trace_rate = 0.01
rare_event_one_in = 25000000
rare_event_is_error = true
retention_days = 21
num_services = 9
peak_to_avg_ratio = 5.5
compression_ratio = 8.0
decision_wait_seconds = 12.0
memory_overhead_factor = 2.0
collector_instances = 6
bill_ingest_pre_sampling = false
[cost]
name = "tempo-on-s3-estimate"
per_gb_ingest = 0.0
per_gb_stored_month = 0.023
per_million_spans = 0.0
fixed_monthly = 400.0
[[strategies]]
kind = "head_probabilistic"
params = { rate = 0.05 }
[[strategies]]
kind = "tail_composite"
params = { baseline_rate = 0.01 }Valid kind values are none, head_probabilistic, head_rate_limit, adaptive,
tail_errors, tail_latency and tail_composite. Recognised params keys are rate and
consistent (head probabilistic), limit_per_service_per_sec (rate limit),
target_traces_per_sec (adaptive) and baseline_rate (all tail policies). A name key
overrides the generated display name.
Unknown fields are rejected rather than ignored, so a typo in a profile field fails loudly instead of silently using the default.
Run python3 -m samplingsim --list-presets for the current values with their reasoning.
They are configurable estimates assembled from public list prices, not vendor quotes.
| Preset | Per GB ingest | Per GB stored/month | Per million spans | Fixed/month |
|---|---|---|---|---|
self-hosted-tempo-s3 |
$0.00 | $0.023 | $0.00 | $250 |
jaeger-elasticsearch |
$0.00 | $0.10 | $0.00 | $600 |
vendor-per-span |
$0.00 | $0.00 | $0.50 | $0 |
vendor-per-gb |
$0.30 | $0.05 | $0.00 | $0 |
free |
$0.00 | $0.00 | $0.00 | $0 |
The two self-hosted presets are dominated by their fixed term, which is the point: below a
certain volume, sampling harder saves you nothing, because you are paying for a cluster
rather than for bytes. The free preset zeroes every price for when you only care about
volumes. For the storage side of that decision see
trace storage backend comparison: Jaeger vs Tempo,
and for how long to keep it,
configuring Jaeger retention policies for compliance.
python3 -m http.server 8000 --directory webThen open http://localhost:8000/. Everything runs in the browser: vanilla ES modules, no
bundler, no CDN, no runtime network access.
- Live controls for every profile field, paired slider and number input.
- Headline tiles for unsampled throughput, errors lost per day, decision-buffer memory and rare-event capture probability.
- A comparison table with the same columns and the same numbers as the CLI, sortable by cost, volume, storage, dropped errors or capture probability.
- Four hand-rolled SVG charts (no chart library): retained span throughput against sampling rate, monthly cost against sampling rate, rare-event capture probability against sampling rate for 1 h / 24 h / 7 d windows, and a stacked storage breakdown by strategy split into error traces and everything else. The first three use a logarithmic rate axis; all four have labelled axes, a crosshair readout, and direct labels or a legend so no series is identified by colour alone.
- Light and dark themes via
prefers-color-scheme, each with its own palette chosen and validated for colour-vision-deficiency separation and contrast against its own surface rather than flipped automatically. - Shareable URLs. The full configuration is encoded in the URL hash in readable
key=valueform, so a pasted link reproduces exactly what you were looking at, and a colleague can see what you changed. "Copy shareable link" puts it on the clipboard. - Export the comparison as CSV or a Markdown table, to file or clipboard. The CSV
columns match
--format csvexactly. - Each strategy card carries a one-line explainer and a link to the article that explains the concept in depth.
Two implementations of the same maths is normally an invitation to drift. This repository handles it with a generated contract:
samplingsim/engine.pyis the source of truth.python3 -m samplingsim.fixtureevaluates a matrix of profiles × cost models × strategies — including deliberately degenerate ones: zero traffic, 0% sampling, 100% sampling, a rate limit far above the offered load — and writes every input and every output field toshared/fixtures.json.tests/test_fixture.pyasserts that the Python engine still reproduces the file byte-for-byte, so changing the engine without regenerating the fixture fails.tests/js/engine.test.mjsreplays every case throughweb/js/engine.jsand compares every leaf value to a relative tolerance of1e-12, so regenerating the fixture without updating the JavaScript engine also fails. It additionally checks that the default profile, the cost presets and the generated strategy display names match.- CI runs both suites plus a job that regenerates the fixture and fails on any diff.
The practical rule when changing the maths: edit samplingsim/engine.py, mirror it in
web/js/engine.js, run python3 -m samplingsim.fixture, then run both test suites.
python3 -m pytest -q # 69 tests
node --test tests/js/*.test.mjs # 17 testsThe Python suite covers:
- Closed-form checks — round-number profiles whose expected outputs were worked out by hand (10 MB/s, 86.4 GB/day, 0.0298 keep fraction, 200 MB of decision buffer).
- Monte-carlo cross-validation of the rare-event maths, in
tests/test_montecarlo.py. The capture probability, the mean time to first capture and the binomial thinning of error traces are each checked against a seeded simulation that does not use the closed form, with tolerances derived from the estimator's standard error (4σ) rather than guessed. The binomial sampler itself is checked for bias first, so a failure in the cross-validation cannot be blamed on the simulator. - Edge cases — 0% and 100% sampling, zero traffic, out-of-range rates clamped into range, invalid profiles rejected, and the assertion that 100% head sampling produces results identical to no sampling.
- CLI behaviour — every output format parsed back and checked, flag precedence over config files, cost preset overrides, sort orders, and unknown config fields rejected.
The JavaScript suite replays the shared fixture and then checks the behaviour the fixture
does not pin: the closed forms of the capture probability, its monotonicity and Poisson
limit, zero-traffic handling without NaN leaking, sort stability, and input validation.
samplingsim/ Python package — the source of truth
profile.py Traffic profile dataclass and validation
strategy.py Strategy definitions and the keep-fraction maths
cost.py Cost model and presets
engine.py The simulation: volume, bandwidth, storage, buffer, cost, probability
format.py Table / CSV / Markdown / JSON / detail rendering
cli.py Argument parsing and config loading
fixture.py Generates shared/fixtures.json
shared/fixtures.json The cross-implementation contract (generated)
web/
index.html The app
css/styles.css Light and dark palettes
js/engine.js JavaScript mirror of the Python engine
js/charts.js Hand-rolled SVG charts
js/state.js URL-hash encoding
js/export.js CSV and Markdown export
js/format.js Number formatting
js/articles.js Article deep links
js/app.js UI wiring
tests/
test_engine.py Closed-form and edge-case checks
test_montecarlo.py Seeded simulation cross-validation
test_fixture.py Python half of the cross-implementation contract
test_cli.py CLI behaviour
js/engine.test.mjs JavaScript half of the contract
examples/ecommerce.toml The worked example above
- It is a steady-state analytic model, not a discrete-event simulator. It will not show you queueing behaviour, back-pressure, retry storms or the shape of a spike; it shows you the averages and the peak those things happen around.
- Peak is a single multiplier. Real traffic has a shape, and a rate limiter's behaviour over a diurnal cycle is not the same as its behaviour at the instantaneous peak. The average and peak keep fractions bracket the answer rather than describing the curve.
- The adaptive strategy is modelled at its target, with no convergence dynamics. A real adaptive sampler overshoots and undershoots as it retunes.
- No per-service breakdown. Rate limits are modelled as if load were spread evenly across services, which it never is; a hot entry point will hit its limit long before the others do.
- Storage costs ignore index and replication overhead beyond what is folded into the preset per-GB prices, and ignore query costs entirely.
- The web app requires an HTTP server, because ES modules cannot be loaded from
file://. This is a browser security rule, not a limitation of the code.
On the site this tool accompanies:
- Choosing between head-based and tail-based sampling
- Head-based vs tail-based sampling cheatsheet
- When to use tail-based sampling for microservices
- Configuring the batch and memory_limiter processors — sizing the collector around the decision buffer
- Trace storage backend comparison: Jaeger vs Tempo
- Configuring Jaeger retention policies for compliance
- Span lifecycle and parent-child relationships — why fragmented traces are so much less useful
MIT — see LICENSE.