Skip to content

volunteer: submission and verification-verdict documents + protocol docs page #4042

Description

@chernistry

Problem

The last two document types are submission ("patch reference + receipt bundle reference") and verification verdict ("gate re-run outcome"). The receipt bundle they both reference already exists and is the most complete precedent in the codebase for what a signed volunteer-program document looks like: src/bernstein/core/security/result_receipt_bundle.py (425 lines, merged as #3870) is a DSSE-wrapped ResultBundle with task: TaskRef, patch, gates: tuple[GateResult, ...], manifest_sha256, chain: ChainLink, and a digest property — signed and offline-verifiable via bernstein receipt verify (src/bernstein/cli/commands/receipt_cmd.py). A "submission" document, read literally, is almost entirely redundant with the receipt bundle it references — the risk in this slice is building a second, parallel copy of fields the bundle already carries (task ref, patch digest) instead of a genuinely thin reference.

Proposal

  • src/bernstein/core/protocols/volunteer/submission.py — a Submission document carrying only what is NOT already inside the receipt bundle: receipt_bundle_digest: str (the bundle's own .digest, i.e. ResultBundle.digest from result_receipt_bundle.py), receipt_bundle_location: str (where a verifier fetches it — a URL, a PR artifact reference, a hub path), task_ref: TaskRef-shaped fields (duplicated deliberately, see "why," not because the bundle's copy isn't trusted), and nothing else. No patch content, no gate logs, no manifest hash duplicated a second time beyond what's needed to route a verifier to the right bundle.
  • src/bernstein/core/protocols/volunteer/verdict.py — a VerificationVerdict document: submission_digest: str (which submission this verdict is about), gate_results: tuple[...] (pass/fail per gate, NOT full logs — the receipt bundle already carries logs; a verdict is the maintainer-facing summary), verifier_keyid: str, recommendation (an enum, not free text — matches volunteer: independent review pass — a second donor reviews every submission #3892's later use of this same document type for a human-legible review recommendation). Both sign via the shared substrate from the first sub-issue.
  • docs/volunteer/protocol.md — written last, once all five document types exist across the three sub-issues of volunteer: transport-neutral protocol documents (project card, worker card, claim, submission) #3883, describing each document's fields, the schema_version policy, and the two conformance projections. This is the one purely-documentation deliverable folded into this slice rather than split into a fourth sub-issue, because an accurate protocol doc needs all five documents to already exist — writing it earlier would mean rewriting it three times.

Why this shape

  • Submission duplicates task_ref on purpose, despite the bundle already carrying one. The receipt bundle's TaskRef (result_receipt_bundle.py's TaskRef dataclass) is signed by the worker, inside the bundle. A submission document is what routes a verifier TO a bundle before they have fetched and verified it — if submission carried no task reference of its own, a maintainer skimming a list of open submissions would have to fetch and verify every bundle just to know which task each one claims to be about, defeating the point of a lightweight routing document. The two copies are allowed to disagree (a lie in the un-verified submission pointing at an honest bundle, or vice versa) — that disagreement is itself useful signal a verifier should surface, not something to prevent by only having one copy.
  • Verdict carries pass/fail summaries, not full gate logs. The receipt bundle already embeds full logs with per-log digests (GateResult.log/log_sha256) specifically so tampering is field-level detectable; duplicating multi-kilobyte log text into a second signed document that a hub or GitHub comment has to render is wasted bytes for no new guarantee — the verdict's job is to be small enough to show inline on a PR.
  • recommendation as an enum, not free text, because volunteer: independent review pass — a second donor reviews every submission #3892 (independent review pass, MVP-stretch) explicitly plans to reuse this exact document type ("No new document types — the review claim and verdict ride the existing protocol layer") for a human reviewer's recommendation. A free-text field here would leave volunteer: independent review pass — a second donor reviews every submission #3892 needing either a second document type (violating its own non-goal) or unstructured text where a maintainer-facing UI wants to filter/sort by outcome.
  • Docs written last, once true, not speculatively first — matches this repository's own stated practice for the volunteer manifest docs (docs/reference/volunteer-manifest.md's worked example section describes the actual shipped .bernstein/volunteer.json, not an aspirational one).

Scope

Does NOT include: claim, project card, or worker card documents (prior sub-issues, this one depends on the shared substrate they also depend on), wiring a verdict document into an actual GitHub check run or PR comment (that is #3880 and #3892 — this issue only defines and signs the document), or the review-task-derivation logic #3892 needs (also out of scope here — #3892 consumes this document type, it does not get built by this issue).


Part of #3883 (volunteer: transport-neutral protocol documents). Depends on the shared-substrate sub-issue. Independent of the project-card/worker-card sub-issue (no shared code between them beyond the substrate both depend on) — the two can be worked in either order or in parallel.

Brief for a coding agent
GOAL
Submission (a thin pointer to an already-signed receipt bundle) and
Verification Verdict (a maintainer-facing pass/fail summary with a
structured recommendation) document types, plus docs/volunteer/protocol.md
once all five document types across #3883's three sub-issues exist.

READ FIRST (in this order, and why)
1. src/bernstein/core/security/result_receipt_bundle.py, whole file again —
   specifically TaskRef (small dataclass, ~4 fields), GateResult (command,
   exit_code, log, log_sha256 property), and ChainLink. Submission's
   task-reference fields should be field-for-field identical in name and
   type to TaskRef's (repo, commit_sha, issue_number) — not a redesigned
   shape that happens to carry similar information. A verifier comparing
   "what the submission claims" against "what the bundle attests" should be
   comparing like-for-like fields, not translating between two schemas
   first.
2. src/bernstein/cli/commands/receipt_cmd.py, whole file (179 lines). This
   is how a receipt bundle gets created and verified today
   (`bernstein receipt create` / `bernstein receipt verify`). Submission is
   downstream of this — a submission document's receipt_bundle_digest field
   is exactly the `digest` value `bernstein receipt verify` prints on
   success (receipt_cmd.py's verify_cmd, around the `click.echo(f"  digest:
   {result.digest}")` line). Do not invent a different digest computation
   for "the bundle as referenced by a submission" — it must be the same
   value `bernstein receipt verify` independently reproduces, or the two
   commands disagree about what a bundle's identity is.
3. The shared substrate module (documents.py) from the first sub-issue of
   #3883 — Submission and VerificationVerdict both build on its canonical-
   bytes/sign/verify helpers, whatever shape they actually shipped as.
4. docs/reference/volunteer-manifest.md, whole file (you've likely read
   this already) — specifically its "A worked manifest: this repository's
   own" section near the end. This is the style docs/volunteer/protocol.md
   should match: concrete, refers to real shipped code and real field
   tables, not aspirational prose. When you write protocol.md, do the same
   — point at the actual merged modules by path, the way this file points
   at `.bernstein/volunteer.json` and `src/bernstein/core/volunteer/manifest.py`.

CURRENT SHAPE (quoted)

result_receipt_bundle.py's TaskRef, the shape Submission's task fields mirror:
    @dataclass(frozen=True, slots=True)
    class TaskRef:
        repo: str
        commit_sha: str
        issue_number: int | None = None

        def to_dict(self) -> dict[str, Any]:
            return {"repo": self.repo, "commit_sha": self.commit_sha, "issue_number": self.issue_number}

result_receipt_bundle.py's digest property, the value Submission points at:
    @property
    def digest(self) -> str:
        """sha256 of the canonical bundle bytes -- the chain anchor successors cite."""
        return _sha256_hex(self.canonical_bytes())

receipt_cmd.py's verify output, the CLI surface that independently computes
the same digest (receipt_cmd.py, inside verify_cmd's success branch):
        if result.ok:
            trust = "pinned key" if pinned else "embedded key (trust on first use)"
            click.echo(f"✓ bundle verifies against {trust}")
            click.echo(f"  keyid:  {result.keyid}")
            click.echo(f"  digest: {result.digest}")

CALL SITES
None outside this new module and its own tests. result_receipt_bundle.py and
receipt_cmd.py are read-only references for this issue — nothing about their
behavior changes; Submission and VerificationVerdict are new leaf modules
that import from result_receipt_bundle.py's TaskRef shape (by matching its
fields, not by importing the dataclass itself, to keep the one-way-dependency
discipline the project-card sub-issue also follows: protocols/volunteer/
depends on core/security and core/volunteer, never the reverse).

EXISTING HELPERS TO REUSE, NOT REINVENT
- Canonical bytes / sign / verify: shared substrate from the first
  sub-issue.
- Bundle digest computation: result_receipt_bundle.py's own
  ResultBundle.digest / _sha256_hex — Submission's receipt_bundle_digest
  field is a STRING carrying a value computed elsewhere (by whoever built
  the bundle), never recomputed by Submission itself. Submission does not
  import ResultBundle or recompute anything; it is a pointer, and a test
  should exist proving it stays that thin (see test matrix).

TEST MATRIX (tests/unit/protocols/volunteer/test_submission.py and
test_verdict.py)

Submission:
1. test_submission_carries_the_bundle_digest_verbatim_without_recomputing_it
   — construct a Submission with an arbitrary-looking (even syntactically
   invalid) digest string and confirm Submission accepts and round-trips it
   unchanged; Submission's job is not to validate that a bundle with that
   digest actually exists or verifies — that is a verifier's job, done by
   fetching and checking the bundle, which is out of this document's scope.
2. test_two_submissions_for_the_same_task_and_bundle_hash_identically
3. test_submission_task_ref_fields_match_result_receipt_bundle_taskrefs_field_names
   — an explicit cross-check (e.g. `assert set(f.name for f in
   dataclasses.fields(Submission)) >= {"repo", "commit_sha",
   "issue_number"}` or equivalent for however you structure it) so a future
   rename of TaskRef's fields in result_receipt_bundle.py is caught here
   instead of silently drifting into two incompatible "task reference"
   shapes across the codebase.

Verdict:
4. test_verdict_recommendation_is_a_closed_enum_not_free_text
   — introspection test, mirrors the worker-card sub-issue's "no open field"
   test; the whole point of the enum decision is that this stays enforced.
5. test_verdict_gate_results_carry_pass_fail_not_full_log_text
   — assert the dataclass has no field capable of holding the kind of
   multi-KB string result_receipt_bundle.py's GateResult.log holds; a
   reasonable implementation is a max-length assertion in the constructor
   plus a test that a long string is refused, or (stronger) a field typed
   as `bool` / an enum rather than `str` for the per-gate outcome, with a
   test that there is no way to construct a verdict carrying log text at
   all.
6. test_a_tampered_verdict_recommendation_fails_signature_verification
   — same tamper-detection discipline as every other document in this
   layer; flip one byte of the signed payload, confirm verify() reports it.
7. test_verdict_references_a_submission_digest_not_a_task_id_directly
   — pins the "which submission this verdict is about" field to point at
   Submission's own digest (chaining verdict -> submission -> bundle), not
   directly at a task id, which would let a verdict be replayed against a
   different submission for the same task.

THE TRAP YOU WOULD HAVE HIT YOURSELF
The obvious-looking implementation copies result_receipt_bundle.py's
TaskRef, GateResult, and patch/log fields wholesale into Submission "for
completeness" — it compiles, the tests you'd naturally write for it pass,
and you end up with a second signed document carrying the same patch text
and full gate logs the receipt bundle already carries and already
tamper-protects. That is not a documentation problem, it is a real cost:
every submission now doubles the bytes a verifier downloads and doubles the
places a future bug in canonicalization could produce two different digests
for what should be one fact. Test 5 above exists specifically to fail if
verdict grows a full-log field; hold submission to the same discipline even
though this brief does not enumerate a matching test for every field — if
you catch yourself typing `patch: str` or `log: str` anywhere in
submission.py or verdict.py, stop and ask whether that data already lives in
the bundle these documents point at.

DECISION TO MAKE, NOT GUESS
Whether `receipt_bundle_location` is a required field or may be empty/absent
when a submission is created before its bundle has a stable fetch location
(e.g. the PR that will carry the bundle as an artifact hasn't opened yet).
If it can be empty, a verdict or a maintainer consuming the submission needs
a defined way to ask "is this submission resolvable yet" that is not "try to
fetch an empty string and get a confusing error." If it must always be
present, document what a caller does in the window between "worker finished
and wants to announce a submission" and "the location actually exists" —
does the caller simply wait to construct the Submission document until the
location is known? Either answer is fine; an unstated one means the two
document types the sibling sub-issues also produce risk making incompatible
assumptions about whether a location is guaranteed present.

VERIFICATION
uv run pytest tests/unit/protocols/volunteer/test_submission.py tests/unit/protocols/volunteer/test_verdict.py -v
uv run ruff check src/bernstein/core/protocols/volunteer/submission.py src/bernstein/core/protocols/volunteer/verdict.py
uv run ruff format --check src/bernstein/core/protocols/volunteer/submission.py src/bernstein/core/protocols/volunteer/verdict.py
uv run mypy src/bernstein/core/protocols/volunteer/submission.py src/bernstein/core/protocols/volunteer/verdict.py
# docs/volunteer/protocol.md is prose; verify only that it renders and that
# every field name it documents matches an actual field on the shipped
# dataclasses (grep the doc's field table against `dataclasses.fields(...)`
# for each of the five types, e.g. in a small throwaway script — do not
# hand-verify five field tables by eye).

Metadata

Metadata

Assignees

No one assigned

    Labels

    discoveryAdvertising, registries, browse and join surfacesenhancementNew feature or requestsecuritySecurity hardeningsize/mup-for-grabsListed on up-for-grabs.net — no commitment, low frictionvolunteerVolunteer workers program

    Type

    No type

    Projects

    Status
    Todo

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions