GOAL
Stand up src/bernstein/core/protocols/volunteer/ with a shared canonical-
serialization + DSSE sign/verify base, one real document type (Claim) built
on it end-to-end, and a conformance-test harness proving a document survives
a round trip through two different projections with an identical hash.
READ FIRST (in this order, and why)
1. src/bernstein/core/security/audit_dsse.py:118-260. The dataclasses
(Subject, Statement, Signature, Envelope) and the pae() function you are
reusing wholesale. Read the whole block, not just signatures — the
comments on Envelope.to_json ("canonical JSON bytes - sorted keys,
comma+colon separators") and on pae() (why PAE prevents a payload-type
downgrade attack) are the reasons the shape looks the way it does, and
you need to preserve those reasons, not just the API surface.
2. src/bernstein/core/security/result_receipt_bundle.py, whole file (425
lines, you have probably already read this for other volunteer work).
This is the nearest complete worked example of "take a plain dataclass,
give it a to_dict()/canonical_bytes()/digest, wrap it in a Statement,
PAE-sign it into an Envelope, and offline-verify with field-level
errors." build_result_bundle (~line 216) and verify_result_bundle
(~line 283) are what your documents.py's shared sign/verify helpers
should look like, generalized from "one bundle type" to "any document
dataclass with a to_dict()."
3. src/bernstein/core/volunteer/manifest.py — specifically
canonical_manifest_bytes/_sort_recursive at the very end of the file, and
the VolunteerManifest dataclass's to_canonical_dict()/digest property
earlier in the file. This is the THIRD independent copy of the same
canonicalization idiom (manifest.py, result_receipt_bundle.py, and
audit_dsse.py each have their own _sort_recursive/_sort_keys_recursive).
Recognize this as the established convention across unrelated leaf
modules — do not "fix" it by making manifest.py or result_receipt_bundle.py
import from your new subpackage. Your shared helper is for the five
documents inside protocols/volunteer/ only.
4. tests/fixtures/a2a/v1_agent_card/_generate.py, whole file (short), and
its module docstring in particular. This is the actual precedent to
copy, closer than anything else in the repo: a signed, canonicalized
document type (the A2A agent card) with a committed, regeneratable
golden-vector corpus — a `valid.json` plus one deliberately-broken
variant per named check (`bad_canonicalization.json`,
`tampered_signatures.json`, `wrong_typ.json`, `unknown_kid.json` all sit
next to it in the same directory). The docstring states the discipline
you must match: "Deterministic: the signing key is derived from a fixed
32-byte seed and every timestamp is pinned, so re-running this script
reproduces byte-identical fixtures." Your claim_v1.json golden vector
(and the sibling sub-issues' project_card_v1.json etc.) should be
generated the same way — a small, checked-in `_generate.py` under
tests/fixtures/protocols/volunteer/, not hand-written JSON that nobody
can regenerate when the schema evolves.
5. tests/integration/acp/conformance/test_acp_vectors.py, whole file, and
the fixture directory it reads (tests/fixtures/acp/conformance/*.jsonl).
A second, weaker precedent for "replay golden vectors from a fixture
directory" — it is testing JSON-RPC message replay, not document
canonicalization, so use it only for the parametrized-test-over-a-
fixture-directory structure, not for how it builds fixtures (point 4
above is the one to actually follow for that).
6. src/bernstein/core/protocols/a2a/__init__.py and acp/__init__.py — both
short. This is the __init__.py convention for a protocols subpackage
(what gets re-exported, module-level docstring style) that
protocols/volunteer/__init__.py should match.
CURRENT SHAPE (quoted)
audit_dsse.py's Envelope, Statement, Subject, Signature dataclasses and pae(),
audit_dsse.py:121-135, 138-157, 161-175, 181-206, 223-245 — already fully
quoted in this issue's own reading above; re-read them there rather than here
so you're looking at line numbers, not a stale copy pasted into a ticket.
result_receipt_bundle.py's canonicalization pair, result_receipt_bundle.py:97-108:
def _sort_recursive(value: Any) -> Any:
"""Reorder dict keys at every depth so canonical JSON is byte-stable."""
if isinstance(value, dict):
return {k: _sort_recursive(value[k]) for k in sorted(value.keys())}
if isinstance(value, list):
return [_sort_recursive(v) for v in value]
return value
def canonical_bytes(payload: dict[str, Any]) -> bytes:
"""Deterministic JSON: recursively sorted keys, compact separators, UTF-8.
Matches :func:`audit_dsse._canonical_json`'s discipline so two serialisations
of the same bundle byte-agree -- the property the determinism test asserts.
"""
return json.dumps(_sort_recursive(payload), sort_keys=True, separators=(",", ":")).encode("utf-8")
result_receipt_bundle.py's build/verify shape to generalize, result_receipt_bundle.py:216-260 (build) and 283-292 (verify signature + first check):
def build_result_bundle(
bundle: ResultBundle,
*,
signing_key: Ed25519PrivateKey,
subject_name: str | None = None,
) -> Envelope:
bundle_dict = bundle.to_dict()
bundle_bytes = canonical_bytes(bundle_dict)
digest = _sha256_hex(bundle_bytes)
subject = Subject(
name=subject_name or f"result-receipt-{bundle.task.commit_sha[:12]}.json",
digest={"sha256": digest},
)
predicate = {
"schema_version": BUNDLE_SCHEMA_VERSION,
"bundle_kind": "result-receipt",
"bundle": bundle_dict,
"chain": bundle.chain.to_dict(),
}
statement = Statement(
subjects=[subject],
predicate_type=RESULT_RECEIPT_PREDICATE_TYPE,
predicate=predicate,
)
payload = canonical_bytes(statement.to_dict())
pae_bytes = pae(DSSE_PAYLOAD_TYPE, payload)
signature = signing_key.sign(pae_bytes)
keyid = keyid_from_public_key(signing_key.public_key())
return Envelope(...)
CALL SITES
None outside this new subpackage and its own tests — protocols/volunteer/ is
brand new. Nothing in manifest.py, sandbox_profile.py, or result_receipt_bundle.py
imports it, and this slice does not change any of those three files.
EXISTING HELPERS TO REUSE, NOT REINVENT
- DSSE envelope wrapping/signing/verification: audit_dsse.py's Envelope,
Statement, Subject, Signature, pae, verify_envelope, keyid_from_public_key,
export_public_key_pem, write_envelope, load_envelope, parse_envelope — all
of them. Nothing about DSSE PAE encoding, signature verification, or
envelope (de)serialization should be written fresh in this subpackage.
- Worker identity: same load-or-create-Ed25519-key pattern as #4036 (the
hub's lease store, if it has merged) uses — install_key.py:53-92's SHAPE,
not its literal install-identity file. If #4036 has already landed a
`load_or_create_worker_key`-equivalent, import and use that one instead of
writing a second one; check before you write it.
TEST MATRIX (tests/unit/protocols/volunteer/, a new directory. Confirmed:
tests/unit/protocols/acp/ and tests/unit/protocols/mcp/ already exist as
sibling per-subpackage test directories — tests/unit/protocols/volunteer/
is a third sibling following an established layout, not a novel choice)
1. test_a_claim_documents_canonical_bytes_are_stable_across_key_order
— build the same Claim from dicts constructed with keys in two different
orders (if your constructor takes a dict) or just assert re-serializing
a loaded Claim reproduces the same digest — mirrors
test_reserialising_a_loaded_manifest_reproduces_its_digest in
tests/unit/volunteer/test_volunteer_manifest.py:99, which is the existing
test in this codebase for exactly this property on a sibling document type.
2. test_a_claim_envelope_verifies_against_the_signing_key
3. test_a_claim_envelope_signed_by_one_key_does_not_verify_against_another
4. test_tampering_any_byte_of_the_claim_payload_fails_verification
— mirrors the DSSE tamper-detection discipline result_receipt_bundle.py's
own module docstring calls out as a hard requirement ("Tampering with any
byte of the patch or any gate log fails verification").
5. test_an_unknown_field_on_a_claim_document_round_trips_into_the_digest
— if you follow the manifest's unknown-field-preservation policy (see the
decision note below on whether you should) — otherwise this test proves
the opposite: an unknown field is rejected outright. Whichever you pick,
write the test that pins that choice down, do not leave it unspecified.
6. test_the_github_projection_and_the_plain_json_projection_of_one_claim_hash_identically
— the literal conformance-test acceptance criterion. Build a Claim, render
it as a "GitHub projection" (your own definition — likely a fenced-code
comment body, since that is how the PoC's claim-etiquette issue (#3873,
unmerged) describes claims living as PR/issue comments) and as plain JSON,
parse each back, and assert both reproduce the same canonical digest.
7. test_golden_vector_claim_json_parses_to_the_documented_digest
— read a committed fixture file (tests/fixtures/protocols/volunteer/claim_v1.json
or similar), parse it, and assert its digest matches a second constant
committed alongside it. This is what keeps a future accidental change to
canonicalization from silently changing every document's hash without a
test noticing — the exact failure mode #3883's "versioning and
forward-compat policy" acceptance criterion exists to prevent.
THE TRAP YOU WOULD HAVE HIT YOURSELF
Every existing canonicalization function in this codebase
(_sort_recursive/_sort_keys_recursive, three independent copies) sorts DICT
keys recursively but does NOT reorder list elements. manifest.py's own
docstring says as much only implicitly (it never lists-sorts gates or
allowed_paths), but result_receipt_bundle.py's GateResult list and audit_dsse's
own module docstring are both silent on list ordering because both callers
happen to build their lists in a stable, deterministic order already (gates
run in file order; subjects are a single-element list in practice). Your
Claim document is fine either way (no lists), but if you generalize the
shared canonicalize helper for the sibling sub-issues' documents (worker
card's capability list, submission's gate-result list) without deciding list-
ordering explicitly, you will reproduce the exact bug class
core/sandbox/selection_receipt.py's own docstring calls out by name: "Lists
are canonically ordered by task_id before serialisation. json.dumps(sort_keys=True)
sorts dict keys, not list elements, so an unsorted candidate/loser list built
in completion order would serialise differently between two concurrent
races." Decide and document, in this slice's shared helper, whether lists
get a canonical sort applied before hashing or whether every document type
is individually responsible for building its own lists in a stable order —
and if it's the latter, say so loudly in the docstring, because the failure
mode is silent until two honest callers produce two different hashes for
what they believe is the same document.
DECISION TO MAKE, NOT GUESS
Do the five protocol documents preserve unknown fields (like
VolunteerManifest.extensions — forward-compatible, unknown fields still bind
to the digest) or reject them outright? This is not a detail — it directly
conflicts with the worker-card sub-issue's security requirement ("worker
card schema must make credential leakage structurally impossible... no
free-form secret-bearing fields"), because an unknown-field bag IS a
free-form field by definition. Decide here, at the shared-substrate level,
whether the answer is "documents preserve unknown fields EXCEPT worker card,
which is closed," or "no document in this protocol layer preserves unknown
fields, full stop, and forward-compatibility happens via schema_version
bumps only." Either is defensible; leaving it to whichever sub-issue gets
picked up first to decide ad hoc is how you get three documents that are
forward-compatible and one that silently drops fields, discovered only when
someone diffs two workers' behavior against the same document.
VERIFICATION
uv run pytest tests/unit/protocols/volunteer/ -v
uv run ruff check src/bernstein/core/protocols/volunteer/
uv run ruff format --check src/bernstein/core/protocols/volunteer/
uv run mypy src/bernstein/core/protocols/volunteer/
Problem
Every signed, content-addressed document already in this codebase reimplements the same three-step pattern from scratch: recursively sort dict keys, serialize to compact-separator UTF-8 JSON, sha256 it.
VolunteerManifest'scanonical_manifest_bytes/_sort_recursive(src/bernstein/core/volunteer/manifest.py, bottom of file),ResultBundle'scanonical_bytes/_sort_recursive(src/bernstein/core/security/result_receipt_bundle.py:97-108), andaudit_dsse's_canonical_json/_sort_keys_recursive(src/bernstein/core/security/audit_dsse.py:260-277) are three independent, near-identical copies. That is the established convention here, not an oversight to "fix" — each signed-document module stays a leaf import with no cross-module dependency on another document's internals. #3883 needs five new document types (project card, worker card, claim, submission, verification verdict) that all need this same treatment plus DSSE signing, and building each from zero would either produce five more independent copies (fine, matches the convention) with no shared conformance-testing discipline, or an ad hoc mix of some sharing code and some not, which would be worse than either extreme.Proposal
A new subpackage,
src/bernstein/core/protocols/volunteer/, alongside the existing protocol subpackages (src/bernstein/core/protocols/a2a/,src/bernstein/core/protocols/acp/). This slice ships:documents.py— a small shared base: aschema_versionconvention, a canonical-bytes helper (one shared copy this time, since these five documents are siblings within one new subpackage rather than modules scattered across the codebase — the "each leaf module carries its own copy" convention applies between unrelated modules like manifest.py and result_receipt_bundle.py, not within one new package purpose-built to hold five sibling document types), and sign/verify functions that call straight intobernstein.core.security.audit_dsse'sEnvelope/Statement/Subject/Signature/pae/verify_envelope/keyid_from_public_key— do not reimplement DSSE wrapping a second time;result_receipt_bundle.py'sbuild_result_bundle/verify_result_bundle(lines ~215-260 and ~283-370) is the exact shape to follow for "wrap a dataclass's canonical dict in a Statement, PAE-sign it, return an Envelope."claim.py— the simplest of the five documents (worker_id,task_id,claimed_at), built end-to-end (schema, canonicalize, sign, verify, golden vector) to prove the shared substrate actually works against a real document rather than an abstraction nobody has used yet.Why this shape
audit_dsse's Envelope/Statement/Subject wrapping wholesale, don't re-derive DSSE.result_receipt_bundle.pyalready proves this composes cleanly for a different document type (see its imports at the top of the file:DSSE_PAYLOAD_TYPE, Envelope, Signature, Statement, Subject, keyid_from_public_key, load_envelope, pae, parse_envelope, verify_envelope, write_envelope— all fromaudit_dsse, nothing reimplemented). Do the same here; a second DSSE implementation for "protocol documents specifically" would be a second thing to keep in sync with the DSSE spec forever.Scope
Does NOT include: project card, worker card, submission, or verification-verdict document types (sibling sub-issues),
docs/volunteer/protocol.md(ships with the last functional slice, once all five document types exist), or wiring the manifest loader to emit/consume a project card (sibling sub-issue — no project card exists yet in this slice).Part of #3883 (volunteer: transport-neutral protocol documents), sliced out as the foundation every other document type builds on. No dependency on anything outside this issue and the already-merged
manifest.py/result_receipt_bundle.py/audit_dsse.py.Brief for a coding agent