Skip to content

fix(run): make run_id an experiment fingerprint, excluding output sinks - #31

Merged
MarouaBoud merged 1 commit into
mainfrom
fix/run-id-fingerprint
Sep 18, 2026
Merged

MarouaBoud merged 1 commit into
mainfrom
fix/run-id-fingerprint

Conversation

@MarouaBoud

Copy link
Copy Markdown
Member

Problem

_run_id hashed the entire config, including report output paths (out, trace_db, live). So two runs of the same experiment that only differ in where their artifacts are written got different run ids — even though they produce byte-identical findings. In practice this is a foot-gun: writing the same run to report.html vs /tmp/r.html yields different ids, which reads as "different runs" when nothing about the experiment changed.

Fix

The run id should identify the experiment — target, population, goals, concurrency, chaos, seed — not the paths its report happens to land in. Exclude the output-sink fields from the fingerprint:

_RUNID_EXCLUDED_SINKS = {"out", "live", "trace_db", "plan_only"}
config.model_dump_json(exclude={"report": _RUNID_EXCLUDED_SINKS})

budget_usd stays in — it can change outcomes via the hard-stop, so it's genuinely part of the experiment.

Why it's safe

run_id is passed to the tracer and report for labeling only; it never feeds the simulation RNG (that's seeded separately from config.seed). Verified empirically: findings are byte-identical with and without this change — only the id string differs. Determinism (NFR-REPRO-01) is unaffected.

Test

test_run_id_is_an_experiment_fingerprint_not_an_output_path pins both halves of the invariant: output sinks must not change the id; the experiment config (seed, population, budget) must. Full suite + ruff + mypy green.

@MarouaBoud
MarouaBoud merged commit fe5ed80 into main Sep 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant