[suffro]

Implements Step 2 of .context/state/DECGUARD_V0.1_CODEX_PLAN.md.

What's new

• Properties. The contract gains properties and fuzz sections, with eight properties:
  • choice and noul: option_order;
  • all types: label_format, irrelevant_context, repeated_context, whitespace, paraphrase;
  • noul: inversion (the answer must swap);
  • score: monotonic.
• Shown-label protocol. Backends get the options reordered or reformatted and answer in those labels. DecGuard maps the answers back to the contract labels before comparing.
• Distribution-aware comparisons. TV, JS, max absolute delta, confidence delta, answer flip, rank change and expected-level delta, with per-case limits plus aggregate violation and flip rates.
• Reproducible runs. A SHA-256 random stream keyed by (seed, property, case), pinned by a test. Failing transformations are shrunk greedily. Cases that cannot be transformed are recorded with a reason.
• Paraphrase providers. identity, file, an optional OpenAI-compatible provider, and entry-point plugins.
• Reports. They gain mode and a properties section. Stored comparisons and violations are checked on load, and report --contract re-applies edited limits.
• CLI.
  • fuzz and test --all;
  • replay re-sends stored failures, checks that the seed regenerates them, and exits 1 if a failure reproduces;
  • diff reports flips, confidence and distribution shifts, calibration/latency/error deltas and segments, and applies regression gates with an implicit max_error_rate_increase: 0.
• Mock backend. position_bias simulates an order-sensitive model.
• Docs. New docs/properties.md, docs/regression.md and docs/ci.md. There is also a decision record for the engine design.

Step 2 acceptance criteria

• Option permutation catches an order-sensitive mock (test_option_permutation_catches_an_order_sensitive_backend, the order_sensitive example backend, and over HTTP).
• A stable mock passes the same test.
• Probability-distribution drift is measured correctly (test_probability_drift_is_measured, test_distribution.py).
• Regression comparison catches controlled degradations (test_regression.py, test_diff_catches_a_regression).
• A stored failure is reproducible (test_stored_failures_replay, plus a CI step).
• Fuzzing is deterministic under a fixed seed (test_fuzzing_is_deterministic_under_a_fixed_seed, pinned RNG).
• Reports are machine-readable (JSON for fuzz, diff and replay).
• CI can fail on declared property violations (exit 1; the CI package job asserts it).
• Integration tests cover choice, noul and score (test_examples_pass_all_checks, inversion and monotonic tests).

Verification

• ruff check, ruff format --check, mypy (strict) and pytest: 245 passed, 1 skipped (the opt-in external test).
• The wheel installs in a clean Python 3.11 venv; validate, test, test --all, fuzz (exit 1 on the flawed backend), replay (exit 1) and diff all run.

🤖 Generated with Claude Code

[suffro]

Step 2 replay semantics are finalized in deab7c0.

• Transformed-case backend errors are now selected as real replay failures.
• Matching stable error kinds reproduce; success or a different error kind does not.
• Replay output preserves the stored error and reports changed error kinds distinctly.
• Focused InvalidResponse replay coverage was added without changing normal violation or reduced-example behavior.

Verification: Ruff lint/format, strict mypy, 246 passed + 1 opt-in external test skipped, clean-wheel smoke, Syngraphe context check, and all 8 GitHub Actions jobs passed.

[suffro]

Final replay edge case fixed in 15ad26f: synthetic original_error pairs remain recorded and counted by property error gates, but are excluded from PropertyRun.failures() and default replay selection. Real transformed backend errors and property violations remain replayable.

Verification: Ruff lint/format, strict mypy, 247 passed + 1 opt-in external test skipped, Syngraphe context check, and all 8 GitHub Actions jobs passed.