feat(delivery): context-scoped delivery ledger, dispatch accounting, checkpoint-entry guard (#353 server) - #356
Merged
Merged
Conversation
…eckpoint-entry guard Server half of #353. §1.1 The delivery ledger is scoped to an agent context ------------------------------------------------------ One session serves many agent contexts: a dispatched worker authenticates against the ORCHESTRATOR's session_index, and several workers can hold that index at once. The three delivery tools therefore take an optional `agent_id` naming the context a payload is delivered TO, and `deliveryScope` keys the ledger on that rather than on the session. A fresh spawn reads an empty ledger and takes full delivery; the same id resumed reads its own prior deliveries and collapses them to markers; a sibling worker is unaffected either way. Omitted, the scope is the session's own agent id, which is the whole walk on a solo session. It scopes the ledger only and never rebinds session.agentId. The invariant this protects: a marker is unreadable to a context that never received the bytes it stands for. `agent_id` is existing vocabulary, not a new designator — `deliveredContent` is keyed by agent id, start_session and dispatch_child both take one, and the corpus already mints a worker agent_id per dispatch and binds it into the spawn stub. This puts it on the wire. No session-schema change. §1.2 Per-call reference opt-in on get_technique / get_resource -------------------------------------------------------------- Both take `bundle: 'reference' | 'full'`, mirroring get_activity, so a resumed worker can dedup technique and resource deliveries as well as its activity bundle. `full: true` overrides the opt-in; the contextMode default path is unchanged. §1.3 Dispatch accounting and delivery magnitude ----------------------------------------------- Two events now measure a dispatch from opposite ends. `activity_usage` carries what one cost and only the orchestrator can supply it, so it arrives on the next_activity that exits the activity — one event per exit. `activity_dispatched` records that a dispatch happened, emitted when the dispatched context first reaches the server: it needs no orchestrator cooperation and counts dispatches rather than exits, so resumed workers, out-of-band workers and abandoned sessions all appear. The fresh/resume discriminator is derived from whether that scope already has a dispatch event for the activity, and is echoed on `_meta.dispatch`. technique_fetched, technique_bundled and resource_fetched each carry `chars` — the full payload size, on both delivery paths — and `delivery: full | unchanged`, so delivered characters and characters saved are both summable from the ledger. §1.4 A checkpoint may not be an activity's first step ----------------------------------------------------- scripts/check-checkpoint-entry.ts, registered in scripts/guards.ts so check:all and CI reach it without editing the workflow file. A dispatch whose first step is a question pays full delivery, yields before executing anything, and has to be resumed to do the work; the question belongs to the preceding activity's tail or to the orchestrator's dispatch precondition. A gate does not exempt it — a gate that is usually true is the same wasted dispatch. The guard found four activities, where the issue named one. Measurement ----------- `npm run bench:dispatch` (new) prices a re-dispatch, where bench:token prices a session mode: per activity, a fresh worker dispatch against the same worker resumed, both fetching the activity payload and every step-bound technique, read from the server's own events. Over 6 work-package activities: 1,115,373 chars on the fresh pass against 403,934 on the resume pass — 63.8% of delivery characters not resent, with 6 fresh and 6 resume dispatches discriminated. The resume figure is not zero because reference delivery ships eager resource bodies that full delivery replaces with resource_refs, so the first resume takes on payload the fresh pass never sent. Tests pin the three properties the corpus relaxation rests on: a fresh scope takes full delivery even when it asks for references, the same scope resumed gets references, and one worker never receives another's markers. Guard fixtures cover both directions, and an empty corpus throws rather than passing vacuously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An authoring rule for definitions, code comments, doc comments and commit subjects: state what the system is and why it holds, in the present tense, without naming the design it replaced. The reasoning is what makes it self-enforcing. A reader a year from now has no memory of the previous design, so naming it makes them learn a dead design just to parse the sentence — and the sentence rots the moment the thing it contrasts against is gone. Both failure modes showed up in this change before the rule was written: a doc comment that explained the ledger by way of the rule it replaced, and a guard header whose worked example was an activity the same change had already fixed, so a reader would have grepped for it and found nothing. Live hazards survive the rule; they are just stated as invariants rather than as history. "A marker is unreadable to a context that never received the bytes" is the design. "This is why the old rule forbade it" is the change. Two narrow exceptions: a commit body is a record of a change and may narrate one, and a measurement that is a tool's reason to exist may be cited with its provenance. The workflow canon already states the same discipline for definition prose, as Document in Positive Present. This gives the server tree its half, in the style section that governs it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he walk Same obligation as #348 §1: the pointer bump and the walk re-baseline land in one commit, so a later snapshot failure separates corpus drift from a code regression. This commit is also why the two halves of #353 cannot land separately. The checkpoint-at-entry guard added here rejects four activities in the corpus this tree pinned before the bump, and one mcp-server fixture reached into `validate` for an unconditional checkpoint that the corpus change relocates. CI runs the guards against the PINNED commit, so the guard and the pointer that satisfies it must arrive together — the corpus reaches the server only through this gitlink, so nothing was ever live on the relaxed rule without the context-scoped ledger underneath it. Snapshot delta is the relocation and nothing else: `local-validation-permission` moves from validate's first step to post-impl-review's tail and no longer sets `mark_progress_na` (validate derives that itself, because commit-and-persist consumes the flag at the COMPLETING activity). One walk row flips warning to valid — a pre-existing off-by-one in manifest attribution, where the manifest submitted on the call ENTERING an activity carries the EXITED activity's steps and is validated against the entered one. Removing validate's entry checkpoint removed one spurious "missing steps in manifest" warning; the equivalent one from post-impl-review's `file-index-table` still stands. Worth its own issue, not widened into this one. The mcp-server fixture retargets at post-impl-review's `file-index-table`, which is unconditional and so still exercises the condition_not_met rejection path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
m2ux
force-pushed
the
fix/353-dispatch-overhead
branch
from
July 30, 2026 07:05
33b28f2 to
4ff7832
Compare
A resource cited both whole and by section delivered its file and those sections as separate entries: distinct ids, distinct ledger keys, no overlap check. The section text shipped twice and drew down the eager budget twice, displacing other content. The file now governs where it lands. Bare ids are ordered ahead of their own anchors, and a section is skipped once its file is in the bundle — including when the file collapsed to an unchanged marker, since that means the context already holds it. Skipping costs the worker nothing: the text is present, and get_resource still serves the anchor alone. Ordering is what makes it safe. A file over the eager cap, or one displaced by the budget, never lands — so its sections must still ship, and they do, because the skip is decided from what actually landed rather than from the citations. Measured on the corpus at workflows a94e92f: review-summary's activity carried 43,781 characters of review-mode where the file is 21,101, and validate-specification carried 3,908 against a 2,638-character rubric. Both were techniques that need most of their resource, where citing sections instead would cost more than the file — the duplication was the bundler's to fix, not the definitions'. Refs #358 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Carries workflows 2951dc2, which states in resource-section-or-whole what this PR's bundler does: a delivered whole resource carries its own sections, so a technique citing both ways receives the file alone. Pointer and corpus stamp both name 2951dc2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…dable Sections 11 and 12 were nine and seven bullets, each bullet a single long sentence carrying mechanism, rationale, config defaults and edge cases at once. A reader looking up one fact had to parse a paragraph to find it. Both sections now open with what the feature does in two sentences, then break into named subsections a reader can jump to: what counts as one agent context, how to turn reference delivery on, what the server remembers, what collapses per call, how blocks inside a technique collapse, how to force a full payload, and what gets measured. Bundling follows the same shape — the budget, which steps qualify, how resources are handled per mode, the response shape, stepping, ledger interplay, fidelity. The opt-in table replaces a bullet that listed three mechanisms and their precedence in one sentence. The budget formula is a code block rather than an inline span. No fact is dropped: 96 of the 99 identifiers, flags, event names, paths and figures in the old text appear verbatim in the new. The other three are formatting — "persistent" and context_mode now read inside the larger context_mode: "persistent", and the formula moved into the code block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three files needed a decision; workflow-tools.ts, the state schema, the site tools page, the walk snapshots and mcp-server.test.ts all auto-merged. AGENTS.md held two versions of "Describe the design, not the change to it" — this branch and #357 wrote it independently, with different carve-outs. The rule now carries both: main's PR and issue bodies alongside this branch's commit body and provenance-bearing measurement, plus the live-hazard-as-invariant guidance and the link to principle 17. CLAUDE.md takes the same union, since this branch had only edited its twin and the two are kept in sync by merging. The worktree-naming rule rides in here too. It was committed to docs/agent-authoring-guidance after #357 had already merged at 618d9ff, so it never reached main and would have been lost with the branch. The workflows pointer moves to 6bc46fa, the corpus tip carrying both #351 and #355, and the stamp names the same commit. binding-fidelity passes: main's detector predicate and reconciled triage arrive with this merge, which is what the 17 stale entries on this branch were waiting for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Server half of #353 (§1.1–§1.4), plus §2.3's pointer bump and walk re-baseline.
Review this first; merge it second. The corpus PR is #355 (
workflow/353-context-scoped-delivery→workflows). The second commit here bumps the gitlink to it, which is what makes the split safe: the corpus reaches the server only through that pointer, so the relaxed delivery rule is never live without the context-scoped ledger underneath it. It is also mechanically necessary — CI runs the guards against the pinned commit, and §1.4's new guard rejects four activities in the corpus this tree pinned before the bump.§1.1 — scope the delivery ledger to a worker context
deliveredHashreadstate.deliveredContent[state.agentId], and a dispatched worker authenticates against the orchestrator'ssession_index. Every worker of a session therefore shared one ledger, and reference delivery on a worker session would have handed worker B unchanged-markers for content worker A received.workers-need-full-deliverywas a correct guard over a ledger that could not distinguish worker contexts.The three delivery tools now take an optional
agent_idnaming the calling agent context, and the ledger keys on that (deliveryScopeinsrc/utils/delivery.ts) instead of the session agent:agent_idresumed → its own ledger → unchanged-references (the win)It scopes the ledger only and never rebinds
session.agentId(asserted on disk in the tests).force-full-after-summarization(bundle: "full"/full: true) remains the escape for a resume that did not preserve context.Convention over invention:
deliveredContentwas already keyed by agent id,start_session/dispatch_childalready take anagent_id, and the corpus already mints a workeragent_idper dispatch and binds it into the spawn stub. Nothing new was named; the existing identity simply now reaches the wire. No session-schema change.§1.2 — per-call reference opt-in on
get_technique/get_resourceBoth gated on
state.contextMode === 'persistent'alone, so a resumed worker could not dedup the 86 technique and 30 resource deliveries that run paid for. Both now acceptbundle: 'reference' | 'full', mirroringget_activity.full: truestill overrides, and thecontextModedefault path is untouched.§1.3 — ledger the invisible third
activity_usagefired 11 times against 33 dispatchesactivity_dispatched— one event per dispatched context arriving,{ agentId, dispatch: "fresh" | "resume", chars }, echoed on_meta.dispatch. The discriminator is derived (does this scope already have a dispatch event for this activity?), so no orchestrator cooperation is required and it cannot be under-reported by an orchestrator that forgets to relay.projectHistory.byTypetherefore already tallies dispatches.technique_fetched/technique_bundled/resource_fetchedcarrychars— the FULL payload size on both delivery paths — anddelivery: "full" | "unchanged". Delivered and saved characters are both summable, so the planning folder's payload figures stop being estimates.agent_id, so its first server call of any kind records the dispatch. The prism run that cost 176,215 tokens and produced CR-0 and B7 would now leave a record without callingget_activityat all.§1.4 — guard: a checkpoint may not be an activity's first step
scripts/check-checkpoint-entry.ts, registered inscripts/guards.tssocheck:alland CI pick it up without editing the workflow file. Awhen/conditiongate does not exempt a first-step checkpoint: a gate that is usually true is the same wasted dispatch and the remedy is the same relocation. It found four violations, not the one the issue named.Verification
The success criteria call for re-running a comparable walk — a ~4h, ~4M-token exercise. Standing in for it, deterministically and in-process:
npm run bench:dispatch(new,scripts/run-dispatch-benchmark.ts) measures the axis this issue is about, whichbench:tokendoes not: per activity, a fresh worker dispatch against the same worker resumed, fetching the activity payload plus every step-bound technique in both passes. Figures are read from the server's own history events, so they are the server's accounting rather than the script's estimate. Over 6work-packageactivities:The resume pass is not zero, and honestly so: reference delivery ships eager resource bodies that full delivery replaces with
resource_refs(#322/#323 T1), so the first resume takes on payload the fresh pass never sent — the 63.8% is net of that.Tests (
+19, all green):tests/reference-delivery.test.ts— a fresh scope gets full delivery even when it asks for references; the same scope resumed gets references; worker B never receives worker A's markers (the actual defect §1.1 fixes);bundle/fullprecedence on technique and resource fetches; the on-disk ledger keys on the passed id withsession.agentIduntouched.tests/fetch-observability.test.ts— fresh-then-resume dispatch events withcharsstrictly decreasing; a second worker reads as its own fresh dispatch; an out-of-band dispatch that only ever fetches a technique records exactly one event, not one per call;chars+deliveryon every delivery event, with the collapsed pass reporting the same magnitude it saved.tests/checkpoint-entry-guard.test.ts— hard zero over the real corpus, plus fixtures in both directions and a gated-checkpoint case; an empty corpus throws rather than passing vacuously.Suite and guards: 757 pass, 18/18 guards. One pre-existing failure is unrelated and reproduces identically on untouched
main—tests/session-crypto.test.tsexpectsEACCESfrommkdir /and this sandbox returnsENOENT.§2.3 — pointer bump and re-baseline (second commit)
workflows→ the #355 corpus commit, withnpm run baseline:stampand the refreshedcorpus-sha.json+snapshot.test.ts.snapin the same commit. The snapshot delta is the checkpoint relocation and nothing else. One walk row flipswarning→valid: that is a pre-existing off-by-one in manifest attribution — the manifest submitted on the call entering an activity carries the exited activity's steps and is validated against the entered one, so removing validate's entry checkpoint removed one spurious "missing steps in manifest" warning. The equivalent warning from post-impl-review'sfile-index-tablestill stands. Diagnosed rather than absorbed; worth its own issue, deliberately not widened into this one.Authoring rule (second commit)
AGENTS.md§ Code and doc style gains "describe the design, not the change to it" — definitions, code comments, doc comments and commit subjects state what the system is and why it holds, in the present tense, without naming the design they replaced. A reader a year from now has no memory of the previous design, so naming it makes them learn a dead design just to parse the sentence, and the sentence rots once the thing it contrasts against is gone.Two failures in this change were the worked examples, both now fixed here:
deliveryScope's doc comment explained the ledger by way of the rule it replaced, and the checkpoint-entry guard header's worked example was an activity this same change had already fixed — a reader would have grepped for it and found nothing. Live hazards survive the rule, stated as invariants rather than history: a marker is unreadable to a context that never received the bytes is the design; this is why the old rule forbade it is the change.The workflow canon already states the same discipline for definition prose as Document in Positive Present; this gives the server tree its half. The corpus PR's definition prose was swept against it too.
Docs
resource_resolution_model.md§11 rewritten around agent-context scoping, with the dispatch-accounting and delivery-magnitude contracts;api-reference.mdtool rows;workflow-fidelity.mdLayer 5 magnitude clause;development.mdgains the dispatch-benchmark section. Generated site regions rebuilt.Not in scope
Trace-token emission, artifact handoff digests, repo-binding waste, packaged execution manifests, and splitting
post-impl-reviewall stay excluded per the issue's own table.🤖 Generated with Claude Code