Skip to content

fix(policy): merge partial data_rules overrides onto conservative defaults - #35

Merged
sturlese merged 1 commit into
mainfrom
fix/bughunt-datarules-merge
Jul 23, 2026
Merged

sturlese merged 1 commit into
mainfrom
fix/bughunt-datarules-merge

Conversation

@sturlese

Copy link
Copy Markdown
Owner

Bug (governance downgrade)

A per-class data_rules override that tightens one axis silently dropped the class's other conservative guards. For example, pinning EU residency on internal data:

policy:
  data_rules:
    internal: { regions: [eu] }   # intent: TIGHTEN (residency)

produced DataRule(regions=['eu'], forbid_training_vendors=False) — the forbid_training_vendors guard flipped from its class default True to False. So an override meant to tighten policy loosened it: internal / confidential / restricted data could now be routed to a vendor that trains on it, with no error and no visible diff. The bug hit even restricted, the fail-closed class (restricted: { models: [x] } dropped its training-vendor guard too).

Root cause

_fill_missing_classes backfilled conservative defaults only for classes entirely absent from the file (setdefault, class granularity). A class present but partial in YAML was taken as a complete atomic DataRule, so unspecified fields fell back to DataRule's permissive field defaults instead of the conservative per-class defaults.

This contradicts the module's own contract: StrictModel exists so "a typo in a governance file must be a loud error, not a silently ignored rule," the validator's comment promises overrides "never silently un-govern," and the existing test is named test_partial_data_rules_keep_conservative_defaults — yet defaults were only kept for other classes, never the one being edited.

Fix

Add a mode="before" validator that merges each per-class override dict onto that class's conservative default ({**default.model_dump(), **override}), so an unspecified field keeps the strict default. Un-governing is still possible but must be explicit (forbid_training_vendors: false) — never by omission. Absent classes are still filled whole by _fill_missing_classes.

Tests

  • The existing partial-override test gains an assertion that the overridden class keeps its own forbid_training_vendors guard.
  • test_partial_override_keeps_the_same_class_training_guard: pinning a region on internal keeps the training-vendor guard.
  • test_data_rule_ungoverning_must_be_explicit: an explicit forbid_training_vendors: false is still honored.

Verified the new assertions fail on main (guard silently dropped) and pass on the branch. Full local gate green: 208 passed, coverage 93.34% (≥85%), ruff clean, offline demo + audit verify intact.

🤖 Found and fixed by an autonomous bughunt iteration.

…aults

_fill_missing_classes backfilled conservative defaults only for data classes
entirely ABSENT from the org file (setdefault, at CLASS granularity). When an
org overrode a class to tighten ONE axis — e.g. `internal: {regions: [eu]}` to
pin residency — pydantic built a fresh DataRule for that class in which every
unspecified field reverted to DataRule's permissive default. Most dangerously
`forbid_training_vendors` flipped from the class default True to False, so an
override meant to TIGHTEN policy silently LOOSENED it: internal / confidential /
restricted data could now reach a vendor that trains on it, with no error and no
visible diff. The bug hit even `restricted`, the fail-closed class.

This contradicts the module's own contract — StrictModel exists so "a typo in a
governance file must be a loud error, not a silently ignored rule", the
validator's comment promises overrides "never silently un-govern", and the
existing test is literally named
`test_partial_data_rules_keep_conservative_defaults`. The defaults were only
kept for OTHER (absent) classes, never for the one being edited.

Add a mode="before" validator that merges each per-class override dict onto that
class's conservative default, so an unspecified field keeps the strict default
(`{**default.model_dump(), **override}`). Un-governing is still possible but must
be written out loud (`forbid_training_vendors: false`), never happen by omission.
Absent classes are still filled whole by _fill_missing_classes. The existing
partial-override test gains an assertion that the overridden class keeps its own
guard, plus new tests for the tightening case and for explicit un-governing.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sturlese

Copy link
Copy Markdown
Owner Author

Adversarial gate: PASS (two independent opus reviewers, both high confidence).

Correctness/intent: merge semantics match documented intent (not atomic-replace) — the module insists absence is visible-not-implicit and governance typos must fail loudly, and the validator/tests codify "never silently un-govern by omission." All construction paths verified stable (defaults, model_validate, YAML round-trip, model_copy, empty override {} → conservative default, unknown class/field still rejected by StrictModel, no shared-mutable-list aliasing). Explicit forbid_training_vendors: false still un-governs.

Security-completeness — verified end-to-end at runtime:

  • main: internal: {regions:[eu]} → forbid_training_vendors=False, the training-vendor model is cleared and the router picks it → hole open.
  • branch: guard stays True, the training-vendor model is excluded, router picks a compliant model → hole closed.

Full suite 208 passed; demo + audit verify intact (the demo org sets the guard explicitly, so the merge is a no-op there — effective policy unchanged).


⚠️ Upgrade note. This is a security tightening: an org that wrote a bare partial override on a sensitive class and unknowingly relied on the old silent loosening will see stricter policy now. In particular restricted: {regions: [...]} now correctly preserves the fail-closed models: [] default, so restricted workflows will route to no model until the org lists an explicit allowlist (previously they silently cleared every registry model). This is the intended conservative behavior — but it can surface as a NoRouteError for such orgs, which should add the intended models:/forbid_training_vendors: fields explicitly.

@sturlese
sturlese merged commit aa19cf6 into main Jul 23, 2026
5 checks passed
@sturlese
sturlese deleted the fix/bughunt-datarules-merge branch July 23, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant