Skip to content

feat(delivery): context-scoped delivery ledger, dispatch accounting, checkpoint-entry guard (#353 server) - #356

Merged
m2ux merged 7 commits into
mainfrom
fix/353-dispatch-overhead
Jul 30, 2026
Merged

feat(delivery): context-scoped delivery ledger, dispatch accounting, checkpoint-entry guard (#353 server)#356
m2ux merged 7 commits into
mainfrom
fix/353-dispatch-overhead

Conversation

@m2ux

@m2ux m2ux commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Server half of #353 (§1.1–§1.4), plus §2.3's pointer bump and walk re-baseline.

Review this first; merge it second. The corpus PR is #355 (workflow/353-context-scoped-deliveryworkflows). The second commit here bumps the gitlink to it, which is what makes the split safe: the corpus reaches the server only through that pointer, so the relaxed delivery rule is never live without the context-scoped ledger underneath it. It is also mechanically necessary — CI runs the guards against the pinned commit, and §1.4's new guard rejects four activities in the corpus this tree pinned before the bump.

§1.1 — scope the delivery ledger to a worker context

deliveredHash read state.deliveredContent[state.agentId], and a dispatched worker authenticates against the orchestrator's session_index. Every worker of a session therefore shared one ledger, and reference delivery on a worker session would have handed worker B unchanged-markers for content worker A received. workers-need-full-delivery was a correct guard over a ledger that could not distinguish worker contexts.

The three delivery tools now take an optional agent_id naming the calling agent context, and the ledger keys on that (deliveryScope in src/utils/delivery.ts) instead of the session agent:

  • fresh spawn → empty ledger → full delivery (unchanged behaviour)
  • same agent_id resumed → its own ledger → unchanged-references (the win)
  • sibling worker → its own empty ledger → unaffected either way
  • omitted → the session's agent, so solo persistent walks keep their exact ledger shape

It scopes the ledger only and never rebinds session.agentId (asserted on disk in the tests). force-full-after-summarization (bundle: "full" / full: true) remains the escape for a resume that did not preserve context.

Convention over invention: deliveredContent was already keyed by agent id, start_session / dispatch_child already take an agent_id, and the corpus already mints a worker agent_id per dispatch and binds it into the spawn stub. Nothing new was named; the existing identity simply now reaches the wire. No session-schema change.

§1.2 — per-call reference opt-in on get_technique / get_resource

Both gated on state.contextMode === 'persistent' alone, so a resumed worker could not dedup the 86 technique and 30 resource deliveries that run paid for. Both now accept bundle: 'reference' | 'full', mirroring get_activity. full: true still overrides, and the contextMode default path is untouched.

§1.3 — ledger the invisible third

Gap Instrument
activity_usage fired 11 times against 33 dispatches activity_dispatched — one event per dispatched context arriving, { agentId, dispatch: "fresh" | "resume", chars }, echoed on _meta.dispatch. The discriminator is derived (does this scope already have a dispatch event for this activity?), so no orchestrator cooperation is required and it cannot be under-reported by an orchestrator that forgets to relay. projectHistory.byType therefore already tallies dispatches.
Delivery events carried no magnitude technique_fetched / technique_bundled / resource_fetched carry chars — the FULL payload size on both delivery paths — and delivery: "full" | "unchanged". Delivered and saved characters are both summable, so the planning folder's payload figures stop being estimates.
Out-of-band dispatches invisible An out-of-band worker mints its own agent_id, so its first server call of any kind records the dispatch. The prism run that cost 176,215 tokens and produced CR-0 and B7 would now leave a record without calling get_activity at all.

§1.4 — guard: a checkpoint may not be an activity's first step

scripts/check-checkpoint-entry.ts, registered in scripts/guards.ts so check:all and CI pick it up without editing the workflow file. A when/condition gate does not exempt a first-step checkpoint: a gate that is usually true is the same wasted dispatch and the remedy is the same relocation. It found four violations, not the one the issue named.

Verification

The success criteria call for re-running a comparable walk — a ~4h, ~4M-token exercise. Standing in for it, deterministically and in-process:

npm run bench:dispatch (new, scripts/run-dispatch-benchmark.ts) measures the axis this issue is about, which bench:token does not: per activity, a fresh worker dispatch against the same worker resumed, fetching the activity payload plus every step-bound technique in both passes. Figures are read from the server's own history events, so they are the server's accounting rather than the script's estimate. Over 6 work-package activities:

freshPass  1,115,373 chars
resumePass   403,934 chars      → 63.8% of delivery characters no longer resent
dispatchEvents { fresh: 6, resume: 6 }

The resume pass is not zero, and honestly so: reference delivery ships eager resource bodies that full delivery replaces with resource_refs (#322/#323 T1), so the first resume takes on payload the fresh pass never sent — the 63.8% is net of that.

Tests (+19, all green):

  • tests/reference-delivery.test.ts — a fresh scope gets full delivery even when it asks for references; the same scope resumed gets references; worker B never receives worker A's markers (the actual defect §1.1 fixes); bundle/full precedence on technique and resource fetches; the on-disk ledger keys on the passed id with session.agentId untouched.
  • tests/fetch-observability.test.ts — fresh-then-resume dispatch events with chars strictly decreasing; a second worker reads as its own fresh dispatch; an out-of-band dispatch that only ever fetches a technique records exactly one event, not one per call; chars + delivery on every delivery event, with the collapsed pass reporting the same magnitude it saved.
  • tests/checkpoint-entry-guard.test.ts — hard zero over the real corpus, plus fixtures in both directions and a gated-checkpoint case; an empty corpus throws rather than passing vacuously.

Suite and guards: 757 pass, 18/18 guards. One pre-existing failure is unrelated and reproduces identically on untouched maintests/session-crypto.test.ts expects EACCES from mkdir / and this sandbox returns ENOENT.

§2.3 — pointer bump and re-baseline (second commit)

workflows → the #355 corpus commit, with npm run baseline:stamp and the refreshed corpus-sha.json + snapshot.test.ts.snap in the same commit. The snapshot delta is the checkpoint relocation and nothing else. One walk row flips warningvalid: that is a pre-existing off-by-one in manifest attribution — the manifest submitted on the call entering an activity carries the exited activity's steps and is validated against the entered one, so removing validate's entry checkpoint removed one spurious "missing steps in manifest" warning. The equivalent warning from post-impl-review's file-index-table still stands. Diagnosed rather than absorbed; worth its own issue, deliberately not widened into this one.

Authoring rule (second commit)

AGENTS.md § Code and doc style gains "describe the design, not the change to it" — definitions, code comments, doc comments and commit subjects state what the system is and why it holds, in the present tense, without naming the design they replaced. A reader a year from now has no memory of the previous design, so naming it makes them learn a dead design just to parse the sentence, and the sentence rots once the thing it contrasts against is gone.

Two failures in this change were the worked examples, both now fixed here: deliveryScope's doc comment explained the ledger by way of the rule it replaced, and the checkpoint-entry guard header's worked example was an activity this same change had already fixed — a reader would have grepped for it and found nothing. Live hazards survive the rule, stated as invariants rather than history: a marker is unreadable to a context that never received the bytes is the design; this is why the old rule forbade it is the change.

The workflow canon already states the same discipline for definition prose as Document in Positive Present; this gives the server tree its half. The corpus PR's definition prose was swept against it too.

Docs

resource_resolution_model.md §11 rewritten around agent-context scoping, with the dispatch-accounting and delivery-magnitude contracts; api-reference.md tool rows; workflow-fidelity.md Layer 5 magnitude clause; development.md gains the dispatch-benchmark section. Generated site regions rebuilt.

Not in scope

Trace-token emission, artifact handoff digests, repo-binding waste, packaged execution manifests, and splitting post-impl-review all stay excluded per the issue's own table.

🤖 Generated with Claude Code

m2ux and others added 3 commits July 30, 2026 08:04
…eckpoint-entry guard

Server half of #353.

§1.1 The delivery ledger is scoped to an agent context
------------------------------------------------------
One session serves many agent contexts: a dispatched worker authenticates against
the ORCHESTRATOR's session_index, and several workers can hold that index at once.
The three delivery tools therefore take an optional `agent_id` naming the context a
payload is delivered TO, and `deliveryScope` keys the ledger on that rather than on
the session. A fresh spawn reads an empty ledger and takes full delivery; the same
id resumed reads its own prior deliveries and collapses them to markers; a sibling
worker is unaffected either way. Omitted, the scope is the session's own agent id,
which is the whole walk on a solo session. It scopes the ledger only and never
rebinds session.agentId.

The invariant this protects: a marker is unreadable to a context that never
received the bytes it stands for.

`agent_id` is existing vocabulary, not a new designator — `deliveredContent` is
keyed by agent id, start_session and dispatch_child both take one, and the corpus
already mints a worker agent_id per dispatch and binds it into the spawn stub. This
puts it on the wire. No session-schema change.

§1.2 Per-call reference opt-in on get_technique / get_resource
--------------------------------------------------------------
Both take `bundle: 'reference' | 'full'`, mirroring get_activity, so a resumed
worker can dedup technique and resource deliveries as well as its activity bundle.
`full: true` overrides the opt-in; the contextMode default path is unchanged.

§1.3 Dispatch accounting and delivery magnitude
-----------------------------------------------
Two events now measure a dispatch from opposite ends. `activity_usage` carries what
one cost and only the orchestrator can supply it, so it arrives on the
next_activity that exits the activity — one event per exit. `activity_dispatched`
records that a dispatch happened, emitted when the dispatched context first reaches
the server: it needs no orchestrator cooperation and counts dispatches rather than
exits, so resumed workers, out-of-band workers and abandoned sessions all appear.
The fresh/resume discriminator is derived from whether that scope already has a
dispatch event for the activity, and is echoed on `_meta.dispatch`.

technique_fetched, technique_bundled and resource_fetched each carry `chars` — the
full payload size, on both delivery paths — and `delivery: full | unchanged`, so
delivered characters and characters saved are both summable from the ledger.

§1.4 A checkpoint may not be an activity's first step
-----------------------------------------------------
scripts/check-checkpoint-entry.ts, registered in scripts/guards.ts so check:all and
CI reach it without editing the workflow file. A dispatch whose first step is a
question pays full delivery, yields before executing anything, and has to be
resumed to do the work; the question belongs to the preceding activity's tail or to
the orchestrator's dispatch precondition. A gate does not exempt it — a gate that
is usually true is the same wasted dispatch. The guard found four activities, where
the issue named one.

Measurement
-----------
`npm run bench:dispatch` (new) prices a re-dispatch, where bench:token prices a
session mode: per activity, a fresh worker dispatch against the same worker
resumed, both fetching the activity payload and every step-bound technique, read
from the server's own events. Over 6 work-package activities: 1,115,373 chars on
the fresh pass against 403,934 on the resume pass — 63.8% of delivery characters
not resent, with 6 fresh and 6 resume dispatches discriminated. The resume figure
is not zero because reference delivery ships eager resource bodies that full
delivery replaces with resource_refs, so the first resume takes on payload the
fresh pass never sent.

Tests pin the three properties the corpus relaxation rests on: a fresh scope takes
full delivery even when it asks for references, the same scope resumed gets
references, and one worker never receives another's markers. Guard fixtures cover
both directions, and an empty corpus throws rather than passing vacuously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An authoring rule for definitions, code comments, doc comments and commit
subjects: state what the system is and why it holds, in the present tense,
without naming the design it replaced.

The reasoning is what makes it self-enforcing. A reader a year from now has no
memory of the previous design, so naming it makes them learn a dead design just
to parse the sentence — and the sentence rots the moment the thing it contrasts
against is gone. Both failure modes showed up in this change before the rule was
written: a doc comment that explained the ledger by way of the rule it replaced,
and a guard header whose worked example was an activity the same change had
already fixed, so a reader would have grepped for it and found nothing.

Live hazards survive the rule; they are just stated as invariants rather than as
history. "A marker is unreadable to a context that never received the bytes" is
the design. "This is why the old rule forbade it" is the change.

Two narrow exceptions: a commit body is a record of a change and may narrate
one, and a measurement that is a tool's reason to exist may be cited with its
provenance.

The workflow canon already states the same discipline for definition prose, as
Document in Positive Present. This gives the server tree its half, in the style
section that governs it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he walk

Same obligation as #348 §1: the pointer bump and the walk re-baseline land in
one commit, so a later snapshot failure separates corpus drift from a code
regression.

This commit is also why the two halves of #353 cannot land separately. The
checkpoint-at-entry guard added here rejects four activities in the corpus this
tree pinned before the bump, and one mcp-server fixture reached into
`validate` for an unconditional checkpoint that the corpus change relocates. CI
runs the guards against the PINNED commit, so the guard and the pointer that
satisfies it must arrive together — the corpus reaches the server only through
this gitlink, so nothing was ever live on the relaxed rule without the
context-scoped ledger underneath it.

Snapshot delta is the relocation and nothing else: `local-validation-permission`
moves from validate's first step to post-impl-review's tail and no longer sets
`mark_progress_na` (validate derives that itself, because commit-and-persist
consumes the flag at the COMPLETING activity). One walk row flips warning to
valid — a pre-existing off-by-one in manifest attribution, where the manifest
submitted on the call ENTERING an activity carries the EXITED activity's steps
and is validated against the entered one. Removing validate's entry checkpoint
removed one spurious "missing steps in manifest" warning; the equivalent one
from post-impl-review's `file-index-table` still stands. Worth its own issue,
not widened into this one.

The mcp-server fixture retargets at post-impl-review's `file-index-table`, which
is unconditional and so still exercises the condition_not_met rejection path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
m2ux and others added 4 commits July 30, 2026 09:12
A resource cited both whole and by section delivered its file and those sections as
separate entries: distinct ids, distinct ledger keys, no overlap check. The section
text shipped twice and drew down the eager budget twice, displacing other content.

The file now governs where it lands. Bare ids are ordered ahead of their own anchors,
and a section is skipped once its file is in the bundle — including when the file
collapsed to an unchanged marker, since that means the context already holds it.
Skipping costs the worker nothing: the text is present, and get_resource still serves
the anchor alone.

Ordering is what makes it safe. A file over the eager cap, or one displaced by the
budget, never lands — so its sections must still ship, and they do, because the skip
is decided from what actually landed rather than from the citations.

Measured on the corpus at workflows a94e92f: review-summary's activity carried
43,781 characters of review-mode where the file is 21,101, and validate-specification
carried 3,908 against a 2,638-character rubric. Both were techniques that need most
of their resource, where citing sections instead would cost more than the file — the
duplication was the bundler's to fix, not the definitions'.

Refs #358

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Carries workflows 2951dc2, which states in resource-section-or-whole what this PR's
bundler does: a delivered whole resource carries its own sections, so a technique
citing both ways receives the file alone.

Pointer and corpus stamp both name 2951dc2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…dable

Sections 11 and 12 were nine and seven bullets, each bullet a single long sentence
carrying mechanism, rationale, config defaults and edge cases at once. A reader looking
up one fact had to parse a paragraph to find it.

Both sections now open with what the feature does in two sentences, then break into
named subsections a reader can jump to: what counts as one agent context, how to turn
reference delivery on, what the server remembers, what collapses per call, how blocks
inside a technique collapse, how to force a full payload, and what gets measured.
Bundling follows the same shape — the budget, which steps qualify, how resources are
handled per mode, the response shape, stepping, ledger interplay, fidelity.

The opt-in table replaces a bullet that listed three mechanisms and their precedence in
one sentence. The budget formula is a code block rather than an inline span.

No fact is dropped: 96 of the 99 identifiers, flags, event names, paths and figures in
the old text appear verbatim in the new. The other three are formatting — "persistent"
and context_mode now read inside the larger context_mode: "persistent", and the formula
moved into the code block.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three files needed a decision; workflow-tools.ts, the state schema, the site tools page,
the walk snapshots and mcp-server.test.ts all auto-merged.

AGENTS.md held two versions of "Describe the design, not the change to it" — this branch
and #357 wrote it independently, with different carve-outs. The rule now carries both:
main's PR and issue bodies alongside this branch's commit body and provenance-bearing
measurement, plus the live-hazard-as-invariant guidance and the link to principle 17.
CLAUDE.md takes the same union, since this branch had only edited its twin and the two
are kept in sync by merging.

The worktree-naming rule rides in here too. It was committed to
docs/agent-authoring-guidance after #357 had already merged at 618d9ff, so it never
reached main and would have been lost with the branch.

The workflows pointer moves to 6bc46fa, the corpus tip carrying both #351 and #355, and
the stamp names the same commit.

binding-fidelity passes: main's detector predicate and reconciled triage arrive with this
merge, which is what the 17 stale entries on this branch were waiting for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant