Skip to content

Derive the PROTECTED CORE roster from the policy doc; gate the decrypt tool - #587

Merged
WilfordGrimley merged 2 commits into
masterfrom
feat/close-ci-enforcement-gaps
Jul 30, 2026
Merged

Derive the PROTECTED CORE roster from the policy doc; gate the decrypt tool#587
WilfordGrimley merged 2 commits into
masterfrom
feat/close-ci-enforcement-gaps

Conversation

@WilfordGrimley

Copy link
Copy Markdown

The confirmed gap

docs/upstreaming/license-provenance.md §2 lists the PROTECTED CORE files — the vote/federation trust anchor. .github/scripts/check_protected_core_license.py had a PROTECTED_CORE_FILES list that was supposed to enforce it. They disagreed.

POLICY lists but CI DID NOT CHECK:
  decrypt-saved-deck-export/decrypt.mjs
  decrypt-saved-deck-export/tests/decrypt.test.mjs

§2 itself said to add them "in the PR that merges #242 (or immediately after), per this section's own 'keep these in sync in the same PR' convention." PR #242 merged (5ddf109c), both files exist on master, and it was never done. Two files the policy declares part of the trust anchor have carried no gate at all since then.

What shipped — not the one-line fix

Adding two strings to the list would have left the actual defect in place: two hand-maintained lists kept in sync by a convention written in prose, which is the same defect class this repo has been closing all week.

check_protected_core_license.py now holds no list. It parses a marker-bounded region in §2 and derives its file set from the doc — which §2 already declared the source of truth. The doc and the check cannot disagree, because there is only one list. Adding a bullet to §2 gates that file with no code change (demonstrated below).

Roster derived: 11 files (was 9 hardcoded).

Two rules that could not have fired on the new entries

Adding the .mjs paths exposed that the checker was Python-shaped throughout:

rule before after
provenance marker regex required a # leader — // PROVENANCE: ..., AGPL-3.0 in a JS file was invisible accepts #, //, * (block-comment continuation)
import walk ast.parse() only; raises SyntaxError on .mjs and returned [] — i.e. silently walked nothing ESM import / export ... from / dynamic import() / require() specifiers; only relative specifiers resolve (a bare specifier is a node: builtin or npm package, out of scope for the same reason the Python side does not scan PyPI metadata)

So the "one-line fix" would have added two entries that looked gated and were not.

A roster check's characteristic failure is to check nothing and pass

Each of these is a hard finding, never a silent skip:

  • roster markers missing, or out of order
  • marker region present but containing no paths
  • a listed path that does not exist on disk (a typo in the doc)
  • a roster entry in a language the lint cannot walk

Doc changes

  • §2's roster is bounded by <!-- PROTECTED-CORE-ROSTER:BEGIN/END --> and states plainly that it is machine-read.
  • The two (+ its test) prose parentheticals are now explicit backticked paths — only backticked spans are machine-read, and federation-hash-tool/tests/test_hash_my_cards.py had never appeared in the doc at all.
  • The stale "Not yet in check_protected_core_license.py... that file only exists on PR PR-6 deck portability: export/import + standalone decrypt tool #242's branch" bullet is replaced by a dated record of the gap it described, kept because the gap was real and lasted.
  • **Prospectively** is deliberately outside the region (nothing to gate yet; an unresolvable path inside would fail the lint).
  • The §2 paragraph describing the CI check now describes what it actually does.

Can it fail? Five demonstrations, each restored to clean after

  1. JS self-marker// PROVENANCE: evil/repo, abc123, AGPL-3.0 prepended to decrypt.mjs:
    ::error file=decrypt-saved-deck-export/decrypt.mjs::PROTECTED CORE file itself carries an AGPL provenance marker
    exit=2
    
    This is the case that was structurally impossible to detect before.
  2. JS transitive import — same marker; the test file is flagged for what it imports:
    ::error file=decrypt-saved-deck-export/tests/decrypt.test.mjs::imports '../decrypt.mjs' (decrypt-saved-deck-export/decrypt.mjs), which carries an AGPL provenance marker
    
  3. BEGIN marker deleted from the doc:
    ::error file=docs/upstreaming/license-provenance.md::protected-core roster markers ... not found (or out of order) in §2. This script derives its file list from that region; without the markers it would check NOTHING and pass, so the missing markers are themselves the finding.
    exit=1
    
  4. Typo in a roster path (vote_consensuss.py):
    ::error file=docs/upstreaming/license-provenance.md::protected-core roster lists 'MPCAutofill/cardpicker/vote_consensuss.py', which does not exist in the repo
    exit=1
    
  5. Derivation is real — added - \MPCAutofill/cardpicker/moderation.py`` to §2's region only (no code change) and AGPL-marked that file:
    ::error file=MPCAutofill/cardpicker/tag_consensus.py::imports 'cardpicker.moderation' (...), which carries an AGPL provenance marker
    ::error file=MPCAutofill/cardpicker/moderation.py::PROTECTED CORE file itself carries an AGPL provenance marker
    exit=2
    

Clean after every restore: protected-core-license: clean (11 files checked, derived from docs/upstreaming/license-provenance.md §2).

Verification

  • .github/scripts/tests/test_check_protected_core_license.py32 tests, OK (was 8). New coverage: JS comment leaders, every JS specifier form, relative/extensionless/bare resolution, JS self-marker and transitive cases, and seven roster-derivation cases (missing doc, missing markers, reversed markers, empty region, non-path backticks ignored, duplicates collapsed, prose outside the region excluded).
  • python3 .github/scripts/check_protected_core_license.py — clean, 11 files.
  • python3 .github/scripts/docs_lint.py --strict — clean; test_docs_lint.py — OK.
  • pre-commit run (ruff/isort/black/mypy/prettier/readme-parity) — passed on commit.

Nothing under MPCAutofill/ changed, so the backend suite was not re-run.

🤖 Generated with Claude Code

https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN

WilfordGrimley and others added 2 commits July 30, 2026 08:38
…t tool

docs/upstreaming/license-provenance.md §2 declared
`decrypt-saved-deck-export/decrypt.mjs` and its test part of the
vote/federation trust anchor, and instructed that both be added to
check_protected_core_license.py's PROTECTED_CORE_FILES "in the PR that
merges #242 (or immediately after)". #242 merged (5ddf109), both files
landed on master, and the CI list was never updated. Two files the
policy calls a trust anchor have carried no gate at all since then.

The one-line fix is to add them to the list. That is not what this does,
because the defect is not the missing entry — it is that two
hand-maintained lists were kept in sync by a convention written in
prose. check_protected_core_license.py now holds NO list: it parses the
marker-bounded roster region in §2 and derives its file set from the
doc, which §2 already declared the source of truth. They cannot disagree
because there is only one list.

Adding the .mjs entries also exposed that neither existing rule could
have fired on them:
  - the provenance-marker regex required a `#` comment leader, so a
    `// PROVENANCE: ..., AGPL-3.0` line in a JS file was invisible. It
    now accepts `#`, `//` and `*`.
  - the import walk was ast.parse()-only, which raises SyntaxError on
    .mjs and returned []. ES-module `import`/`export ... from` /
    dynamic `import()` / `require()` specifiers are now extracted, with
    only relative specifiers resolving (a bare specifier is a node:
    builtin or an npm package — out of scope for the same reason the
    Python side does not scan PyPI metadata).

A roster check's characteristic failure is to check nothing and pass, so
a missing/out-of-order marker, an empty region, a listed path that does
not exist, and a roster entry in an unwalkable language are each HARD
findings rather than silent skips.

§2 also spells out the two "(+ its test)" prose parentheticals as
explicit paths, since only backticked spans are machine-read, and
replaces the stale "not yet in the CI script" bullet with a dated record
of the gap it described.

Roster derived: 11 files (was 9 hardcoded).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
The roster is derived from license-provenance.md (covered by `docs/**`),
but the FILES it gates were not all trigger paths.
`decrypt-saved-deck-export/**` had none: a PR pasting AGPL-marked code
into the decrypt tool touches no doc and no cardpicker module, so the
protected-core lint would not have run on it until the weekly cron.
`federation-hash-tool/**` was already listed; this is its counterpart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
@WilfordGrimley
WilfordGrimley force-pushed the feat/close-ci-enforcement-gaps branch from 74a1fda to b8a05e0 Compare July 30, 2026 08:39
@WilfordGrimley
WilfordGrimley merged commit 4ff100f into master Jul 30, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant