Skip to content

feat: add deterministic RNG checkpoints - #189

Open
anantgar wants to merge 3 commits into
SakanaAI:mainfrom
anantgar:codex/rng-checkpointing
Open

feat: add deterministic RNG checkpoints#189
anantgar wants to merge 3 commits into
SakanaAI:mainfrom
anantgar:codex/rng-checkpointing

Conversation

@anantgar

@anantgar anantgar commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add atomic clean checkpoints for Shinka-owned random streams and async runner state.
  • Add seeded strict, if_available, and reseed resume modes plus Ctrl-C/programmatic checkpoint requests.
  • Bring randomized SQLite selection under the checkpointed Python RNG.

Why

A seed recreates the start of a random stream, but the previous resume path did not continue its exact position. Clean checkpoints let long campaigns stop and resume from a verified, drained boundary.

Testing

  • uv run ruff check tests --exclude tests/file.py
  • uv run mypy --follow-imports=skip --ignore-missing-imports tests/test_*.py tests/conftest.py
  • uv run --with pytest-cov pytest -q tests -m "not requires_secrets" --cov=shinka --cov-report=term-missing --cov-report=xml:coverage.xml: 921 passed, 7 skipped, 1 deselected
  • uv run --group docs mkdocs build --strict

Core evolution pipeline evidence

The local deterministic headless fixture ran with seed 20260830, one proposal/evaluation/DB worker, and two dynamically selected models. The baseline ran uninterrupted to 8 programs; the comparison checkpointed at 4 and strictly resumed to 8. The normalized generation trace matched exactly for parent, archive/top-k inspirations, island, model, code, score, and correctness, and the resume audit recorded deterministic_resume=true.

Reproduce with:

uv run pytest -q tests/test_headless_provider.py::test_seeded_checkpoint_resume_matches_uninterrupted_evolution

Risks and compatibility

  • Existing runs keep best-effort resume by default; strict mode is opt-in.
  • Checkpoints use pickle and must only be loaded from trusted result directories.
  • The guarantee covers Shinka-owned randomness at a clean boundary, not external model responses or live in-flight work.

Docs and UI

Checkpoint configuration, CLI usage, resume modes, and guarantee boundaries are documented. No UI changes.

@anantgar anantgar changed the title feat: add clean RNG checkpoints for deterministic resume feat: add deterministic RNG checkpoints Aug 30, 2026
@anantgar
anantgar marked this pull request as ready for review August 30, 2026 22:37
Move runner checkpoint helpers into checkpointing.py and hash SQLite tables instead of per-column watermarks so the PR stays smaller without changing resume behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant