From 266d4ee7cdf3b4a1ee8b354400a15053dbd74f9d Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Fri, 4 Sep 2026 22:05:53 -0400 Subject: [PATCH 1/7] rfc: multi-blob add Batch upload protocol: /blob/batch/add plans placement for M blobs in one invocation and re-delegates the node verbs to the agent under policy-bound delegations; /blob/batch/commit relays node-signed accept receipts. Two upload-service round trips per batch, two per node, no node changes. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 361 +++++++++++++++++++++++++++++++++ 1 file changed, 361 insertions(+) create mode 100644 rfcs/2026-09-multi-blob-add.md diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md new file mode 100644 index 0000000..b47bc86 --- /dev/null +++ b/rfcs/2026-09-multi-blob-add.md @@ -0,0 +1,361 @@ +# RFC: Multi-blob add + +Status: Experimental + +## Authors + +- [Hannah Howard](https://github.com/hannahhoward) + +## Motivation + +Uploading one blob today costs the client three POSTs to the upload service plus a receipt poll, and costs the upload service two invocations against the storage node, all serial: `/blob/add`, an HTTP PUT of the bytes to the node, `/ucan/conclude` carrying a self-issued `/http/put` receipt, then `GET /receipt/{cid}` polled at one-second granularity until the `/blob/accept` receipt appears. An object of M blobs pays that chain M times. Client-side parallelism hides some of the latency but none of the message count, and the poll adds up to a second per blob after the bytes have already landed. + +Two of those messages carry no information. The `/http/put` receipt is signed with a key derived from the blob digest, so anyone who knows the digest can mint it — sprue's own source comments that it proves nothing — and its only job is to trigger the accept. The receipt poll exists only because the accept completes out of the client's sight, even though it runs synchronously inside the conclude request. + +The clients that drive volume know every digest before the first network call. Ingot spools a PutObject to disk and shards it before uploading; any client that pre-shards or erasure-codes an object holds its full digest list up front. Nothing in the protocol lets them say so. + +This RFC proposes a batch upload protocol: + +- `/blob/batch/add` — announce M blobs in one invocation; the upload service plans placement for all of them and returns, per blob, a selected node plus delegations that let the agent invoke that node directly. +- The agent drives the node legs itself: one container of `/blob/allocate` invocations per node, the PUTs in parallel, one container of `/blob/accept` invocations per node. The node verbs are today's verbs, unchanged. +- `/blob/batch/commit` — return the node-signed accept receipts; the upload service verifies them and registers the blobs. +- `/blob/batch/extend` — replacement placements when a node fails and the spares are exhausted. + +Cost for an M-blob object on N nodes: two round trips to the upload service, two to each node, M PUTs in parallel. No `/http/put`, no `/ucan/conclude`, no receipt poll. + +## Goals + +- Client↔upload-service round trips are independent of blob count: two per batch. +- No storage-node changes. Nodes keep serving `/blob/allocate` and `/blob/accept` exactly as today; only the invoker changes. The protocol deploys against existing piri. +- Every node-side action remains traceable to a client-signed intent: the allocate's `cause` names the client's batch invocation, and the delegation policy binds it. +- Partial success is first-class. Blobs succeed and fail independently; commit is idempotent and resumable, so a straggler commits later under the same batch. +- The self-issued `/http/put` receipt and the receipt poll are removed from the flow entirely. + +## Concepts + +### Roles + +| Name | Description | +| ----- | ---------------------------------------------------------------------------- | +| Agent | A client of the Forge network holding delegations from a space (e.g. Ingot). | +| Sprue | The Forge network upload service. | +| Piri | A Forge network storage node (a.k.a. storage provider). | +| Space | A namespace (cryptographic key pair) that owns content. A bucket is a space. | + +### The third delegation regime + +The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: **re-delegated provider-rooted**. + +Sprue already holds the provider's registration delegation for `/blob/allocate` and `/blob/accept`. For each blob it places, sprue re-delegates those two commands to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: + +```jsonc +// Policy on the /blob/allocate delegation (dag-json for readability) +[ + ["==", ".space", "did:key:zSpace"], + ["==", ".blob.digest", { "/": { "bytes": "EiCcvAfD..." } }], + ["==", ".blob.size", 1048576], + ["==", ".cause", { "/": "bafy...batchaddtask" }] +] +``` + +The agent invokes on the node under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can invoke only the announced digest, at the announced size, for the named space and cause, at the node sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. + +The regime grants the agent nothing it does not already have in effect: today's flow produces the same two invocations on the same node at sprue's discretion. Here sprue pre-authorizes precisely that pair and steps off the data path. + +The baseline is one delegation pair (allocate, accept) per blob. For large batches sprue MAY compact to one pair per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per blob, letting the agent retire its own parked pieces early instead of waiting for allocation expiry (see [Compatibility](#compatibility)). + +### Batch lifecycle + +1. **Plan** — `/blob/batch/add`. Sprue selects a node per blob over its provider directory in one pass, mints the delegations, and returns placements. Sprue contacts no node and writes no placement state; blobs already registered in the space are returned as such, with no placement. +2. **Node legs** — for each node holding placements, the agent sends one container batching that node's `/blob/allocate` invocations. Each allocate receipt returns the upload address (or none, when the node already holds the bytes — the warm path, unchanged). The agent PUTs all pieces in parallel, then sends one container per node batching the `/blob/accept` invocations. Accept behaves exactly as today: the node verifies presence, mints its location claim, publishes it synchronously, and enqueues the PDP pipeline. +3. **Commit** — `/blob/batch/commit` relays the node-signed accept receipts. Sprue verifies them and registers each blob against the space. +4. **Repair the tail** — a failed or stalled PUT retries against a spare placement with no sprue call. Only exhausting the spares costs a `/blob/batch/extend`. + +### Transport + +Batching travels as ucantone container contents: a single HTTP exchange carries a container holding any number of invocations, and receipts ride container metadata the same way. This stack has no ucanto-style effects field, and none is needed. + +## Capabilities + +### `/blob/batch/add` + +* Issuer: Agent +* Audience: Sprue +* Subject: The space + +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns one placement per blob plus spares, with the minted delegations carried in the response container. Digests already registered in the space are listed in `registered` and receive no placement. + +The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. + +Idempotent: re-announcing a digest is safe — the node keys allocations by `(digest, space)`, and re-registration at commit tolerates the existing entry. + +#### Arguments + +**IPLD schema** + +```ipldsch +type BlobEntry struct { + digest Bytes # multihash of the blob + size Int # size in bytes +} + +type BatchAddArguments struct { + blobs [BlobEntry] +} +``` + +
+Go syntax + +```go +type BatchAddArguments struct { + Blobs []blob.Blob `cborgen:"blobs"` // libforge commands/blob Blob{Digest, Size} +} +``` +
+ +e.g. + +```jsonc +// Encoded as dag-json for readability +{ + "iss": "did:key:agent", + "aud": "did:web:up.forge.example.com", + "sub": "did:key:space", + "cmd": "/blob/batch/add", + "args": { + "blobs": [ + { "digest": { "/": { "bytes": "EiCcvAfD..." } }, "size": 1048576 }, + { "digest": { "/": { "bytes": "EiDpQrst..." } }, "size": 524288 } + ] + } +} +``` + +#### Result + +**IPLD schema** + +```ipldsch +type Placement struct { + digest Bytes # multihash of the blob + provider String # DID of the selected storage node + proofs [Link] # delegations in the response container: /blob/allocate, /blob/accept, optionally /blob/reject +} + +type BatchAddOK struct { + placements [Placement] # primary placement per blob + spares [Placement] # alternates, usable on failure without a further sprue call + registered [Bytes] # digests already registered in this space; no placement issued +} +``` + +
+Go syntax + +```go +type Placement struct { + Digest multihash.Multihash `cborgen:"digest"` + Provider did.DID `cborgen:"provider"` + Proofs []cid.Cid `cborgen:"proofs"` +} + +type BatchAddOK struct { + Placements []Placement `cborgen:"placements"` + Spares []Placement `cborgen:"spares"` + Registered []multihash.Multihash `cborgen:"registered"` +} +``` +
+ +### `/blob/batch/commit` + +* Issuer: Agent +* Audience: Sprue +* Subject: The space + +Closes out a batch. The request container MUST carry, per committed blob, the `/blob/accept` invocation and its node-signed receipt; the arguments reference each receipt by link. Sprue verifies each pair (see [Commit verification](#commit-verification)), registers each verified blob against the space with the batch-add task as its cause, persists the relayed invocations and receipts to its agent store, and returns a per-blob outcome. + +A batch never has to commit whole. Commit registers whatever verifies and names the rest; a later commit under the same `cause` picks up stragglers. Registration tolerates an existing entry, so overlapping or repeated commits converge. + +Idempotent: committing an already-registered blob MUST succeed. + +#### Arguments + +**IPLD schema** + +```ipldsch +type CommitEntry struct { + digest Bytes # multihash of the blob + accept Link # the node-signed /blob/accept receipt, carried in the request container +} + +type BatchCommitArguments struct { + cause Link # the /blob/batch/add task this commit closes + blobs [CommitEntry] +} +``` + +
+Go syntax + +```go +type CommitEntry struct { + Digest multihash.Multihash `cborgen:"digest"` + Accept cid.Cid `cborgen:"accept"` +} + +type BatchCommitArguments struct { + Cause cid.Cid `cborgen:"cause"` + Blobs []CommitEntry `cborgen:"blobs"` +} +``` +
+ +#### Result + +**IPLD schema** + +```ipldsch +type CommitFailure struct { + digest Bytes + reason String # named error, e.g. ReceiptNotFound, InvalidReceipt, UnknownProvider +} + +type BatchCommitOK struct { + registered [Bytes] # digests now (or already) registered in the space + failed [CommitFailure] # entries that did not verify; re-commit after repair +} +``` + +
+Go syntax + +```go +type CommitFailure struct { + Digest multihash.Multihash `cborgen:"digest"` + Reason string `cborgen:"reason"` +} + +type BatchCommitOK struct { + Registered []multihash.Multihash `cborgen:"registered"` + Failed []CommitFailure `cborgen:"failed"` +} +``` +
+ +### `/blob/batch/extend` + +* Issuer: Agent +* Audience: Sprue +* Subject: The space + +Requests replacement placements for blobs whose primary and spare placements have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh placements and spares with fresh delegations — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch commits under one cause. + +Idempotent: extending a digest that has since been accepted or registered simply returns no placement for it, listed in `registered`. + +#### Arguments + +**IPLD schema** + +```ipldsch +type ExtendEntry struct { + digest Bytes + exclude [String] # provider DIDs to avoid +} + +type BatchExtendArguments struct { + cause Link # the original /blob/batch/add task + blobs [ExtendEntry] +} +``` + +
+Go syntax + +```go +type ExtendEntry struct { + Digest multihash.Multihash `cborgen:"digest"` + Exclude []did.DID `cborgen:"exclude"` +} + +type BatchExtendArguments struct { + Cause cid.Cid `cborgen:"cause"` + Blobs []ExtendEntry `cborgen:"blobs"` +} +``` +
+ +#### Result + +`BatchAddOK`, as for [`/blob/batch/add`](#blobbatchadd). + +### The node legs: `/blob/allocate` and `/blob/accept`, unchanged + +The storage-node verbs, their argument types and their results are exactly today's. What changes is the invoker and two argument conventions: + +- **`cause`** on `/blob/allocate` is the `/blob/batch/add` task CID (bound by the delegation policy), so every allocation row a node writes names the client invocation that motivated it. +- **`_put`** on `/blob/accept` is vestigial on this path: there is no `/http/put` task to await, because the node itself verified presence when the bytes arrived. The field carries the corresponding `/blob/allocate` task link for wire compatibility, and implementations MUST NOT dereference it. Deprecating the field is left to a future revision of the accept capability. + +The warm path is preserved: an allocate receipt with no upload address means the node already holds the bytes, the agent skips that PUT, and the blob's accept rides the accept container with the rest. + +## Commit verification + +Commit verification is stateless with respect to placement: sprue does not need to remember what it minted, because the proof of placement travels with the commit. For each entry, sprue: + +1. Resolves the receipt link in the request container and the invocation it `Ran`. +2. Checks the invocation: command `/blob/accept`, arguments matching the entry's digest and the invocation subject's space, and a proof chain that terminates in a delegation **sprue itself signed** — the placement delegation minted at batch-add, which names the provider, digest, size and cause. Sprue verifying its own signature is the placement check. +3. Checks the receipt: issued and signed by the provider DID that is the invocation's subject, `Ran` matching, and a success result. +4. Registers the blob with `cause` = the batch-add task, and persists the invocation and receipt to the agent store. + +An entry failing any check lands in `failed` with a named reason and blocks nothing else. + +Implementation note: the container accessor for receipts is a linear scan. Matching M receipts to M entries naively is O(M²); index the container's receipts by `Ran()` once before the loop. + +## Failure handling and idempotency + +- A failed or stalled PUT retries against a spare placement; no sprue call. Exhausted spares cost one `/blob/batch/extend`. +- A node that accepted before failing still yields a valid receipt; those blobs commit. Only bytes that never reached an accept re-place. +- Allocations that never receive bytes, and spares that go unused, expire on the node's existing allocation-expiry hygiene, with no message required. +- Every capability in this RFC is idempotent: re-adding, re-committing and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". + +## Security considerations + +- **Attenuation is the containment.** The delegation policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can produce exactly the allocations sprue decided, nowhere else, and only until expiry. +- **No new agent trust.** The agent gains no authority it could not already exercise through `/blob/add`; sprue's placement discretion is unchanged, exercised at plan time instead of invocation time. +- **No new node exposure.** The node serves the same verbs under a chain rooted in its own registration delegation. It cannot tell, and need not care, whether sprue or the agent carried the invocation. +- **Removing a fake signal.** The digest-derived `/http/put` receipt let anyone with a digest assert an upload happened. This protocol replaces it with the node-signed accept receipt as the only evidence of storage — a signature that actually attests something. +- **Replay.** Delegations expire; invocations carry nonces; allocation is keyed `(digest, space)`, so a replayed allocate reproduces the same row rather than a new liability. +- **Resource bounds.** A batch is an amplification lever: one invocation makes sprue run placement and mint delegations for M blobs. Sprue SHOULD enforce a configurable cap on blobs per batch and account batch adds against the space's quota before minting. + +## Limits + +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-blob delegation baseline puts two to three signed delegations per blob, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand blobs. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged. + +## Compatibility + +- **Storage nodes**: no change. The chain provider → sprue → agent validates under existing subject-plus-policy checking. +- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, upload, commit, repeat. +- **Blob removal (RFC 2026-07)**: receipt-chain routing for accepted blobs is preserved, because commit persists the relayed `/blob/accept` invocation — whose subject is the provider — into the agent store, and registration's `cause` reaches it. Parked blobs differ: sprue never saw the allocate, so `/blob/abort` cannot route for a batch-parked blob. The backstop is node-side allocation expiry, which reconciles parked pieces with no message at all; the optional `/blob/reject` delegation in the placement proofs gives an agent early reclaim directly against the node. +- **Read-on-write**: preserved. The node still publishes its location claim synchronously inside accept, and registration completes inside commit, so a successful commit still implies the blob is resolvable. Index and upload registration (`/index/add`, `/upload/add`) follow commit exactly as they follow the accept receipt today, and are out of scope here. + +## Future work + +- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node containers to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it, and the delegation regime is the same. +- **Deprecate `_put`** on `/blob/accept` once no invoker populates it meaningfully. + +## Example flow + +An object of M blobs placed across N nodes: + +| # | Leg | What moves | +|---|-----|------------| +| 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | +| 2 | sprue → agent | `BatchAddOK`: M placements + spares, delegations in the container | +| 3 | agent → each node, parallel | one container batching that node's `/blob/allocate` invocations; receipts carry upload addresses | +| 4 | agent → nodes, parallel | M HTTP PUTs; stragglers retried against spares | +| 5 | agent → each node, parallel | one container batching that node's `/blob/accept` invocations; receipts carry location claims and PDP promises | +| 6 | agent → sprue | one `/blob/batch/commit` relaying the accept receipts | +| 7 | sprue → agent | `BatchCommitOK`: per-blob outcomes; the S3 200 follows | + +Round trips: two to sprue per batch and two per node, against roughly four to sprue **per blob** (three POSTs plus at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: 42 control exchanges, all parallel across nodes, instead of ~600, serial per blob. From 739957a837eef5248e073d764197bf21337094b5 Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Fri, 4 Sep 2026 23:14:21 -0400 Subject: [PATCH 2/7] rfc: multi-blob add: conclude-based winner report The batch pair concludes with /ucan/conclude generalized to many receipts: per winner, the node-signed /blob/allocate receipt plus a client-minted /http/put receipt. Allocate-only re-delegation; accept stays sprue-invoked behind a durable conclusion journal; today's add->allocate->put->accept chain shape is preserved. Applies the verification pass: corrected current-flow counts, conclude authorization stated as a strengthening, placement replica bound, no-freshness delegation check at conclude, quota holds, and house-format sections. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 275 ++++++++++++++++++++------------- 1 file changed, 170 insertions(+), 105 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index b47bc86..a854714 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -1,4 +1,4 @@ -# RFC: Multi-blob add +# RFC: Forge multi-blob add Status: Experimental @@ -8,28 +8,28 @@ Status: Experimental ## Motivation -Uploading one blob today costs the client three POSTs to the upload service plus a receipt poll, and costs the upload service two invocations against the storage node, all serial: `/blob/add`, an HTTP PUT of the bytes to the node, `/ucan/conclude` carrying a self-issued `/http/put` receipt, then `GET /receipt/{cid}` polled at one-second granularity until the `/blob/accept` receipt appears. An object of M blobs pays that chain M times. Client-side parallelism hides some of the latency but none of the message count, and the poll adds up to a second per blob after the bytes have already landed. +Uploading one blob today costs the client two POSTs to the upload service, an HTTP PUT of the bytes to the storage node, and a receipt poll, and costs the upload service two invocations against the storage node, all serial: `/blob/add`, the PUT, `/ucan/conclude` carrying a self-issued `/http/put` receipt, then `GET /receipt/{cid}` until the `/blob/accept` receipt appears — at least one more round trip per blob, and a second of added latency whenever the receipt is not ready on the first try. An object of M blobs pays that chain M times. Client-side parallelism hides some of the latency but none of the message count. -Two of those messages carry no information. The `/http/put` receipt is signed with a key derived from the blob digest, so anyone who knows the digest can mint it — sprue's own source comments that it proves nothing — and its only job is to trigger the accept. The receipt poll exists only because the accept completes out of the client's sight, even though it runs synchronously inside the conclude request. - -The clients that drive volume know every digest before the first network call. Ingot spools a PutObject to disk and shards it before uploading; any client that pre-shards or erasure-codes an object holds its full digest list up front. Nothing in the protocol lets them say so. +The clients that drive volume know every digest before the first network call. Ingot spools a PutObject to disk and shards it before uploading; any client that pre-shards or erasure-codes an object holds its full digest list up front. Nothing in the protocol lets them say so. And the protocol has no vocabulary for racing an upload: one node is selected per blob, and if it stalls, the whole blob waits. This RFC proposes a batch upload protocol: -- `/blob/batch/add` — announce M blobs in one invocation; the upload service plans placement for all of them and returns, per blob, a selected node plus delegations that let the agent invoke that node directly. -- The agent drives the node legs itself: one container of `/blob/allocate` invocations per node, the PUTs in parallel, one container of `/blob/accept` invocations per node. The node verbs are today's verbs, unchanged. -- `/blob/batch/commit` — return the node-signed accept receipts; the upload service verifies them and registers the blobs. -- `/blob/batch/extend` — replacement placements when a node fails and the spares are exhausted. +- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate nodes plus delegations that let the agent invoke `/blob/allocate` on those nodes directly. +- The agent drives allocation and upload itself: one container of `/blob/allocate` invocations per node, then PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land. Over-allocation and long-tail cancellation become placement policy rather than failure handling. +- `/ucan/conclude`, generalized to many receipts. For each blob that landed, the client concludes with the node-signed `/blob/allocate` receipt and the `/http/put` receipt. Sprue verifies the pair, drives `/blob/accept` at each winning node (one container per node), registers the blobs, and returns the accept receipts in the conclude response. +- `/blob/batch/extend` — replacement candidates when a blob exhausts its set. -Cost for an M-blob object on N nodes: two round trips to the upload service, two to each node, M PUTs in parallel. No `/http/put`, no `/ucan/conclude`, no receipt poll. +Cost for an M-blob object: two client↔sprue round trips per batch, one agent↔node round trip per node plus the PUTs, and one sprue↔node round trip per winning node. No receipt poll: the conclude response carries the accept receipts. ## Goals -- Client↔upload-service round trips are independent of blob count: two per batch. -- No storage-node changes. Nodes keep serving `/blob/allocate` and `/blob/accept` exactly as today; only the invoker changes. The protocol deploys against existing piri. +- Client↔upload-service round trips are independent of blob count: two per batch, plus one `/blob/batch/extend` exchange per repair round. +- No storage-node changes. Nodes keep serving `/blob/allocate` and `/blob/accept` exactly as today; only the allocate invoker changes. The protocol deploys against existing piri. +- Accept authority stays with the upload service. Accept is the liability-creating verb — the node mints and publishes a location claim, enqueues PDP, and the blob leaves the parked state that expiry can reclaim — so it remains invoked by the service, adjacent to the registration that accounts for it. +- The upload service drops out of the per-blob upload sequence. It mints capabilities without contacting a node, and touches nodes only at accept, after the client reports what landed. - Every node-side action remains traceable to a client-signed intent: the allocate's `cause` names the client's batch invocation, and the delegation policy binds it. -- Partial success is first-class. Blobs succeed and fail independently; commit is idempotent and resumable, so a straggler commits later under the same batch. -- The self-issued `/http/put` receipt and the receipt poll are removed from the flow entirely. +- Partial success is first-class. Blobs succeed and fail independently; conclusion is idempotent and resumable, so a straggler concludes later under the same batch. +- The receipt chain keeps today's shape — `add → allocate → put → accept` — so everything that walks receipts (replay, removal routing) is unchanged. ## Concepts @@ -41,12 +41,11 @@ Cost for an M-blob object on N nodes: two round trips to the upload service, two | Sprue | The Forge network upload service. | | Piri | A Forge network storage node (a.k.a. storage provider). | | Space | A namespace (cryptographic key pair) that owns content. A bucket is a space. | +| Ingot | An S3 facade typically co-located with a Forge Piri node. | -### The third delegation regime - -The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: **re-delegated provider-rooted**. +### The third delegation regime: re-delegated allocate -Sprue already holds the provider's registration delegation for `/blob/allocate` and `/blob/accept`. For each blob it places, sprue re-delegates those two commands to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: +The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: sprue re-delegates **`/blob/allocate` only** to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: ```jsonc // Policy on the /blob/allocate delegation (dag-json for readability) @@ -58,18 +57,39 @@ Sprue already holds the provider's registration delegation for `/blob/allocate` ] ``` -The agent invokes on the node under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can invoke only the announced digest, at the announced size, for the named space and cause, at the node sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. +The agent invokes allocate on the node under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can allocate only the announced digest, at the announced size, for the named space and cause, at the nodes sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. + +Allocate is the safe verb to hand over: it is idempotent, keyed `(digest, space)`, and an allocation that never receives bytes is parked state the node's existing expiry reclaims. The worst an agent can do with an allocate delegation is create allocations sprue already decided to permit, which then expire. `/blob/accept` is deliberately **not** re-delegated: acceptance creates obligations that do not expire, so it stays a service-invoked verb, issued only after the client reports success and in the same step that registers the blob. + +The baseline is one allocate delegation per blob per candidate node. For large batches sprue MAY compact to one delegation per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per placement, letting the agent retire its own parked pieces early instead of waiting for allocation expiry; this is safe because reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). + +### Over-allocation and the winner report + +Placement returns more candidates than a blob needs. The agent allocates at its candidates, PUTs to as many as it likes, and stops when enough land — the race is client policy, from "primary first, spare on failure" to "start all, cancel at first." A candidate whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and is an instant winner. -The regime grants the agent nothing it does not already have in effect: today's flow produces the same two invocations on the same node at sprue's discretion. Here sprue pre-authorizes precisely that pair and steps off the data path. +The network learns the outcome from the client, in receipt form. For each blob that landed, the client holds two artifacts: -The baseline is one delegation pair (allocate, accept) per blob. For large batches sprue MAY compact to one pair per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per blob, letting the agent retire its own parked pieces early instead of waiting for allocation expiry (see [Compatibility](#compatibility)). +- the **node-signed `/blob/allocate` receipt** — real evidence that the winning node precommitted to the piece, and the routing for everything that follows: the allocate invocation names the provider (its subject), the space, the digest and the cause, and its proof chain terminates in the delegation sprue itself minted; +- the **`/http/put` receipt** — the completion event, in the protocol's existing conclusion idiom. The put invocation is signed with a key derived from the blob digest, so it attests nothing cryptographically; it is the client's assertion that the bytes landed, and the protocol treats it as exactly that. The real check is the accept that follows: a false conclusion just produces a failed accept at the node. + +Losing candidates need no message at all: their parked allocations, and any bytes from PUTs cancelled after completion, expire node-side. + +### The client-minted `/http/put` + +Today sprue builds the `/http/put` invocation during `/blob/add`, deriving an ed25519 key from the blob digest and embedding it in the invocation metadata so the client can sign the receipt. In this protocol the agent performs the allocate, so the agent builds the put invocation itself — same derivation, same shape: + +- key: the multihash's 32-byte digest as an ed25519 seed; issuer = audience = subject = the derived `did:key`; +- arguments: the blob and a `destination` await linking the agent's own `/blob/allocate` task; +- receipt: self-signed with the derived key on PUT success — immediately, with no PUT, when the allocate receipt carried no upload address. + +Nothing about the artifact's strength changes — anyone holding the digest could always mint it, which is the deliberate design — only where it is minted. The chain `batch add → allocate → put → accept` is exactly today's chain shape, and `/blob/accept`'s `_put` field awaits a real put task, as today. ### Batch lifecycle -1. **Plan** — `/blob/batch/add`. Sprue selects a node per blob over its provider directory in one pass, mints the delegations, and returns placements. Sprue contacts no node and writes no placement state; blobs already registered in the space are returned as such, with no placement. -2. **Node legs** — for each node holding placements, the agent sends one container batching that node's `/blob/allocate` invocations. Each allocate receipt returns the upload address (or none, when the node already holds the bytes — the warm path, unchanged). The agent PUTs all pieces in parallel, then sends one container per node batching the `/blob/accept` invocations. Accept behaves exactly as today: the node verifies presence, mints its location claim, publishes it synchronously, and enqueues the PDP pipeline. -3. **Commit** — `/blob/batch/commit` relays the node-signed accept receipts. Sprue verifies them and registers each blob against the space. -4. **Repair the tail** — a failed or stalled PUT retries against a spare placement with no sprue call. Only exhausting the spares costs a `/blob/batch/extend`. +1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the allocate delegations, and returns placements. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. +2. **Allocate and race** — one container of `/blob/allocate` invocations per candidate node; PUTs in parallel; cancel at enough. The agent mints a put receipt per winner. +3. **Conclude** — one `/ucan/conclude` carrying every winner's receipt pair. Sprue verifies the pairs, coalesces winners by provider, sends one `/blob/accept` container per winning node in parallel, registers each accepted blob with the batch-add task as its cause, and returns the accept receipts and per-blob outcomes in the response. +4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and concludes later under the same batch. ### Transport @@ -83,24 +103,24 @@ Batching travels as ucantone container contents: a single HTTP exchange carries * Audience: Sprue * Subject: The space -Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns one placement per blob plus spares, with the minted delegations carried in the response container. Digests already registered in the space are listed in `registered` and receive no placement. +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob with the minted delegations carried in the response container. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. -Idempotent: re-announcing a digest is safe — the node keys allocations by `(digest, space)`, and re-registration at commit tolerates the existing entry. +Idempotent: re-announcing a digest is safe — allocation is keyed `(digest, space)` on the node, and registration at conclusion tolerates the existing entry. #### Arguments **IPLD schema** ```ipldsch -type BlobEntry struct { +type Blob struct { digest Bytes # multihash of the blob size Int # size in bytes } type BatchAddArguments struct { - blobs [BlobEntry] + blobs [Blob] } ``` @@ -109,7 +129,7 @@ type BatchAddArguments struct { ```go type BatchAddArguments struct { - Blobs []blob.Blob `cborgen:"blobs"` // libforge commands/blob Blob{Digest, Size} + Blobs []blob.Blob `cborgen:"blobs"` // reuses libforge commands/blob Blob{Digest, Size} } ``` @@ -139,13 +159,13 @@ e.g. ```ipldsch type Placement struct { digest Bytes # multihash of the blob - provider String # DID of the selected storage node - proofs [Link] # delegations in the response container: /blob/allocate, /blob/accept, optionally /blob/reject + provider String # DID of the candidate storage node + proofs [Link] # delegations in the response container: /blob/allocate, optionally /blob/reject } type BatchAddOK struct { - placements [Placement] # primary placement per blob - spares [Placement] # alternates, usable on failure without a further sprue call + placements [Placement] # primary candidate per blob + spares [Placement] # additional candidates; race or hold in reserve, client's choice registered [Bytes] # digests already registered in this space; no placement issued } ``` @@ -168,31 +188,43 @@ type BatchAddOK struct { ``` -### `/blob/batch/commit` +### `/ucan/conclude`, generalized * Issuer: Agent * Audience: Sprue * Subject: The space -Closes out a batch. The request container MUST carry, per committed blob, the `/blob/accept` invocation and its node-signed receipt; the arguments reference each receipt by link. Sprue verifies each pair (see [Commit verification](#commit-verification)), registers each verified blob against the space with the batch-add task as its cause, persists the relayed invocations and receipts to its agent store, and returns a per-blob outcome. +The existing conclusion command, extended to carry many receipts. The arguments gain an optional plural field; a server implementing this RFC MUST accept both forms, and the legacy single-receipt flow is untouched. + +The multi-receipt form also tightens authorization, deliberately. Today's conclude is effectively self-authorized — the client issues it with itself as subject, and the handler checks only that the named receipt corresponds to a stored invocation. The multi form MUST carry the space as subject, proven by the agent's space-rooted chain, and registration MUST bind to the space the placement delegation pins, which MUST equal the invocation subject. This closes conclusion of another space's uploads by anyone who learns a task CID. -A batch never has to commit whole. Commit registers whatever verifies and names the rest; a later commit under the same `cause` picks up stragglers. Registration tolerates an existing entry, so overlapping or repeated commits converge. +For each concluded `/http/put` receipt, the request container MUST make the winner's pair resolvable: the put invocation, and — via its `destination` await — the `/blob/allocate` invocation and its node-signed receipt. The handler resolves the pair from the request container; resolution from sprue's agent store remains for the legacy single form, whose artifacts sprue itself wrote. -Idempotent: committing an already-registered blob MUST succeed. +Sprue dispatches each receipt whose `Ran` command has a registered conclusion handler; other receipts in the container are context. The `/http/put` handler, per conclusion: + +1. follows `destination` to the allocate invocation and receipt; +2. verifies the allocate invocation's proof chain terminates in a **placement delegation sprue itself signed**, whose audience is the concluding agent and whose policy pins the provider, space, digest, size and cause — and that the pinned space equals the conclude subject. Sprue verifying its own signature is the placement check, and it requires no stored state. Verification checks signature and policy bindings, **not** freshness: the node-signed allocate receipt attests that the full chain, expiry included, validated at execution time. Provider registration and liveness are checked at accept time; +3. verifies the node's signature on the allocate receipt; +4. enforces the placement bound: at most the placement policy's replica count of providers — one, unless placement says otherwise — is accepted per `(digest, space)`. The first verified pair wins; surplus pairs return a `SurplusWinner` outcome with no accept, and registration records the accepted provider. This bound, not delegation revocation, is what caps winners — an extend's exclusions revoke nothing; +5. invokes `/blob/accept` at the provider — the verification the protocol actually relies on: the node confirms it holds the bytes. Within one conclude request sprue SHOULD coalesce accepts by provider into one container per node, sent in parallel, each under a mandatory per-node deadline so no stalled node delays the response; +6. on accept success, registers the blob against the space with the batch-add task as `cause`, and persists the relayed invocations and receipts to its agent store. + +The multi-receipt form returns per-conclusion outcomes, with the accept receipts in the response container — this is what retires the receipt poll. A conclusion failing any check lands in the outcome list with a named reason and blocks nothing else. + +A batch never has to conclude whole. Conclusion registers whatever verifies; a later conclude under the same batch picks up stragglers. Registration and acceptance tolerate repeats, so overlapping or repeated conclusions converge. Clients SHOULD conclude within the delegation lifetime: node-side allocation expiry can reap unconcluded bytes, and the node's deadline is not visible to sprue. + +Idempotent: concluding an already-accepted blob MUST succeed. + +Implementation note: the container accessor for receipts is a linear scan. Matching M conclusions to their container artifacts naively is O(M²); index the container's contents by link and by `Ran()` once before the loop. #### Arguments **IPLD schema** ```ipldsch -type CommitEntry struct { - digest Bytes # multihash of the blob - accept Link # the node-signed /blob/accept receipt, carried in the request container -} - -type BatchCommitArguments struct { - cause Link # the /blob/batch/add task this commit closes - blobs [CommitEntry] +type ConcludeArguments struct { + receipt optional Link # legacy single form + receipts optional [Link] # the /http/put receipts being concluded } ``` @@ -200,31 +232,50 @@ type BatchCommitArguments struct { Go syntax ```go -type CommitEntry struct { - Digest multihash.Multihash `cborgen:"digest"` - Accept cid.Cid `cborgen:"accept"` +type ConcludeArguments struct { + Receipt *cid.Cid `cborgen:"receipt,omitempty"` + Receipts []cid.Cid `cborgen:"receipts,omitempty"` } +``` + -type BatchCommitArguments struct { - Cause cid.Cid `cborgen:"cause"` - Blobs []CommitEntry `cborgen:"blobs"` +e.g. + +```jsonc +// Encoded as dag-json for readability; the referenced put receipts, +// put invocations, allocate invocations and allocate receipts travel +// in the same container +{ + "iss": "did:key:agent", + "aud": "did:web:up.forge.example.com", + "sub": "did:key:space", + "cmd": "/ucan/conclude", + "args": { + "receipts": [ + { "/": "bafyreia...putrcpt1" }, + { "/": "bafyreib...putrcpt2" } + ] + } } ``` - #### Result +The legacy single form keeps its current result. The multi form returns: + **IPLD schema** ```ipldsch -type CommitFailure struct { - digest Bytes - reason String # named error, e.g. ReceiptNotFound, InvalidReceipt, UnknownProvider +type ConcludeOutcome struct { + receipt Link # the concluded /http/put receipt + registered Bool + accept optional Link # the node-signed /blob/accept receipt, in the response container + reason optional String # named error, e.g. InvalidAllocateReceipt, UnknownPlacement, + # SurplusWinner, AcceptRefused, AcceptTimeout } -type BatchCommitOK struct { - registered [Bytes] # digests now (or already) registered in the space - failed [CommitFailure] # entries that did not verify; re-commit after repair +type BatchConcludeOK struct { + outcomes [ConcludeOutcome] } ``` @@ -232,14 +283,15 @@ type BatchCommitOK struct { Go syntax ```go -type CommitFailure struct { - Digest multihash.Multihash `cborgen:"digest"` - Reason string `cborgen:"reason"` +type ConcludeOutcome struct { + Receipt cid.Cid `cborgen:"receipt"` + Registered bool `cborgen:"registered"` + Accept *cid.Cid `cborgen:"accept,omitempty"` + Reason string `cborgen:"reason,omitempty"` } -type BatchCommitOK struct { - Registered []multihash.Multihash `cborgen:"registered"` - Failed []CommitFailure `cborgen:"failed"` +type BatchConcludeOK struct { + Outcomes []ConcludeOutcome `cborgen:"outcomes"` } ``` @@ -250,7 +302,7 @@ type BatchCommitOK struct { * Audience: Sprue * Subject: The space -Requests replacement placements for blobs whose primary and spare placements have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh placements and spares with fresh delegations — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch commits under one cause. +Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh candidates with fresh delegations — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch concludes under one cause. Idempotent: extending a digest that has since been accepted or registered simply returns no placement for it, listed in `registered`. @@ -292,70 +344,83 @@ type BatchExtendArguments struct { ### The node legs: `/blob/allocate` and `/blob/accept`, unchanged -The storage-node verbs, their argument types and their results are exactly today's. What changes is the invoker and two argument conventions: - -- **`cause`** on `/blob/allocate` is the `/blob/batch/add` task CID (bound by the delegation policy), so every allocation row a node writes names the client invocation that motivated it. -- **`_put`** on `/blob/accept` is vestigial on this path: there is no `/http/put` task to await, because the node itself verified presence when the bytes arrived. The field carries the corresponding `/blob/allocate` task link for wire compatibility, and implementations MUST NOT dereference it. Deprecating the field is left to a future revision of the accept capability. +The storage-node verbs, their argument types and their results are exactly today's. What changes: -The warm path is preserved: an allocate receipt with no upload address means the node already holds the bytes, the agent skips that PUT, and the blob's accept rides the accept container with the rest. +- **`/blob/allocate`** is invoked by the agent under the re-delegation, with `cause` = the `/blob/batch/add` task CID (bound by the delegation policy), so every allocation row a node writes names the client invocation that motivated it. +- **`/blob/accept`** is invoked by sprue under its registration delegation, as today, with `_put` awaiting the client-minted `/http/put` task from the concluded pair. -## Commit verification +## Registration durability -Commit verification is stateless with respect to placement: sprue does not need to remember what it minted, because the proof of placement travels with the commit. For each entry, sprue: +Accept at the node and registration at sprue are two steps, and the gap between them is where accepted-but-unregistered state — a node holding non-expiring, PDP-enqueued bytes no space is registered for — would otherwise leak: an accept that times out after the node committed, a sprue crash between accept and registration, a client that disconnects mid-conclude. The protocol closes the gap by making completion sprue's obligation rather than the request's: -1. Resolves the receipt link in the request container and the invocation it `Ran`. -2. Checks the invocation: command `/blob/accept`, arguments matching the entry's digest and the invocation subject's space, and a proof chain that terminates in a delegation **sprue itself signed** — the placement delegation minted at batch-add, which names the provider, digest, size and cause. Sprue verifying its own signature is the placement check. -3. Checks the receipt: issued and signed by the provider DID that is the invocation's subject, `Ran` matching, and a success result. -4. Registers the blob with `cause` = the batch-add task, and persists the invocation and receipt to the agent store. +- Sprue MUST journal each verified conclusion durably **before** invoking accept, and MUST drive the journaled pair through accept and registration to completion regardless of the request — the handler runs to completion on client disconnect, and a reconciliation sweep re-drives journaled pairs orphaned by a crash or timeout. +- The outcome vocabulary keeps the two failure meanings apart. `AcceptRefused` means the node rejected the accept: the blob is not stored there, and re-placing via `/blob/batch/extend` is the right response. `AcceptTimeout` means the outcome is unknown and sprue owns reconciliation: the client MUST NOT re-place the blob on this signal — re-concluding later is safe and returns the settled outcome. -An entry failing any check lands in `failed` with a named reason and blocks nothing else. - -Implementation note: the container accessor for receipts is a linear scan. Matching M receipts to M entries naively is O(M²); index the container's receipts by `Ran()` once before the loop. +With the journal, a verified conclusion either completes or is retried by sprue itself; no client behavior, node stall, or crash strands an accept unregistered beyond the sweep interval. ## Failure handling and idempotency -- A failed or stalled PUT retries against a spare placement; no sprue call. Exhausted spares cost one `/blob/batch/extend`. -- A node that accepted before failing still yields a valid receipt; those blobs commit. Only bytes that never reached an accept re-place. -- Allocations that never receive bytes, and spares that go unused, expire on the node's existing allocation-expiry hygiene, with no message required. -- Every capability in this RFC is idempotent: re-adding, re-committing and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". +- A failed or stalled PUT races another candidate; no sprue call. An exhausted candidate set costs one `/blob/batch/extend`. +- An `AcceptRefused` outcome sends the blob back through extend and a later conclude; an `AcceptTimeout` outcome is sprue's to reconcile (see [Registration durability](#registration-durability)). +- Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. +- Every capability in this RFC is idempotent: re-adding, re-concluding and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". ## Security considerations -- **Attenuation is the containment.** The delegation policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can produce exactly the allocations sprue decided, nowhere else, and only until expiry. +- **Attenuation.** The allocate delegation's policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can produce exactly the parked allocations sprue decided to permit, nowhere else, and only until expiry. +- **Accept authority never moves.** Accept — the step that mints a published location claim, enqueues PDP, and exits the expiry-reclaimable state — is invoked only by sprue, only after a client conclusion, and its completion is journaled (see [Registration durability](#registration-durability)), so no client can manufacture an accepted-but-unregistered blob. - **No new agent trust.** The agent gains no authority it could not already exercise through `/blob/add`; sprue's placement discretion is unchanged, exercised at plan time instead of invocation time. -- **No new node exposure.** The node serves the same verbs under a chain rooted in its own registration delegation. It cannot tell, and need not care, whether sprue or the agent carried the invocation. -- **Removing a fake signal.** The digest-derived `/http/put` receipt let anyone with a digest assert an upload happened. This protocol replaces it with the node-signed accept receipt as the only evidence of storage — a signature that actually attests something. -- **Replay.** Delegations expire; invocations carry nonces; allocation is keyed `(digest, space)`, so a replayed allocate reproduces the same row rather than a new liability. -- **Resource bounds.** A batch is an amplification lever: one invocation makes sprue run placement and mint delegations for M blobs. Sprue SHOULD enforce a configurable cap on blobs per batch and account batch adds against the space's quota before minting. +- **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and the protocol treats it as one. The node-signed allocate receipt is the signed artifact, and accept succeeding at the node is the verification the protocol relies on. +- **Replay.** Delegations expire; invocations carry nonces; allocation is keyed `(digest, space)`, so a replayed allocate reproduces the same row rather than a new liability; re-conclusion and re-registration converge, and the conclude subject binding stops cross-space conclusion of a leaked pair. +- **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint delegations for M blobs, and one conclude makes it verify and fan out accepts for M winners. Sprue SHOULD enforce configurable caps on blobs per batch and conclusions per conclude. Quota SHOULD be taken as a hold with a TTL equal to the delegation expiry, released on expiry and reconciled at registration — an outright debit leaks on abandoned batches, and a bare check over-commits across parallel batches. ## Limits -The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-blob delegation baseline puts two to three signed delegations per blob, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand blobs. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged. +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-candidate delegation baseline puts one or two signed delegations per candidate, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand candidates. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. ## Compatibility -- **Storage nodes**: no change. The chain provider → sprue → agent validates under existing subject-plus-policy checking. -- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, upload, commit, repeat. -- **Blob removal (RFC 2026-07)**: receipt-chain routing for accepted blobs is preserved, because commit persists the relayed `/blob/accept` invocation — whose subject is the provider — into the agent store, and registration's `cause` reaches it. Parked blobs differ: sprue never saw the allocate, so `/blob/abort` cannot route for a batch-parked blob. The backstop is node-side allocation expiry, which reconciles parked pieces with no message at all; the optional `/blob/reject` delegation in the placement proofs gives an agent early reclaim directly against the node. -- **Read-on-write**: preserved. The node still publishes its location claim synchronously inside accept, and registration completes inside commit, so a successful commit still implies the blob is resolvable. Index and upload registration (`/index/add`, `/upload/add`) follow commit exactly as they follow the accept receipt today, and are out of scope here. +- **Storage nodes**: no change. The provider → sprue → agent allocate chain validates under existing subject-plus-policy checking. +- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, race, conclude, repeat. +- **`/ucan/conclude`**: the plural field is additive; existing single-receipt clients and the legacy flow are untouched. +- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept` — has today's shape, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md#receipt-chain-routing)'s receipt-chain routing work unchanged for concluded blobs. Never-concluded blobs differ: sprue never saw their allocates, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry, which reconciles parked pieces with no message at all; the optional `/blob/reject` delegation in the placement proofs gives an agent early reclaim directly against the node. +- **Read-on-write**: preserved. The node still publishes its location claim synchronously inside accept, and registration completes inside the conclude request, so a successful conclusion still implies the blob is resolvable. Index and upload registration (`/index/add`, `/upload/add`) follow conclusion exactly as they follow the accept receipt today, and are out of scope here. ## Future work -- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node containers to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it, and the delegation regime is the same. -- **Deprecate `_put`** on `/blob/accept` once no invoker populates it meaningfully. +- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it, and the delegation regime is the same. ## Example flow -An object of M blobs placed across N nodes: +An object of M blobs raced across N candidate nodes: | # | Leg | What moves | |---|-----|------------| | 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | -| 2 | sprue → agent | `BatchAddOK`: M placements + spares, delegations in the container | -| 3 | agent → each node, parallel | one container batching that node's `/blob/allocate` invocations; receipts carry upload addresses | -| 4 | agent → nodes, parallel | M HTTP PUTs; stragglers retried against spares | -| 5 | agent → each node, parallel | one container batching that node's `/blob/accept` invocations; receipts carry location claims and PDP promises | -| 6 | agent → sprue | one `/blob/batch/commit` relaying the accept receipts | -| 7 | sprue → agent | `BatchCommitOK`: per-blob outcomes; the S3 200 follows | - -Round trips: two to sprue per batch and two per node, against roughly four to sprue **per blob** (three POSTs plus at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: 42 control exchanges, all parallel across nodes, instead of ~600, serial per blob. +| 2 | sprue → agent | `BatchAddOK`: candidates per blob, allocate delegations in the container | +| 3 | agent → each node, parallel | one container batching that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — warm, instant winner) | +| 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; mint a put receipt per winner | +| 5 | agent → sprue | one `/ucan/conclude` carrying every winner's allocate + put receipt pair | +| 6 | sprue → each winning node, parallel | one container batching that node's `/blob/accept` invocations | +| 7 | sprue → agent | `BatchConcludeOK`: per-blob outcomes, accept receipts in the container; the S3 200 follows | + +Round trips: two to sprue per batch, one per node from the agent, one per winning node from sprue — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 42 control exchanges, all parallel across nodes, instead of ~500, serial per blob. Byte PUTs are excluded from both counts. + +## Open questions + +- Default and maximum sizes for the batch, conclude and candidate-set caps. +- Whether `/blob/reject` delegations ship in placement proofs by default or on request. +- The per-node accept deadline and the reconciliation sweep interval. +- Whether the per-provider `or`-compacted delegation form ships in v1 or waits for a batch size that needs it. + +## Evaluation criteria + +- Control exchanges for an M-blob object drop from ~5M to 2 + 2N, measured on ingot's sharded-object upload. +- End-to-end upload latency for a multi-blob object improves by at least the removed poll and serialization overhead, measured against the current guppy/ingot path. +- No growth in stranded state: accepted-but-unregistered blobs stay at zero under fault injection (accept timeout, sprue restart mid-conclude, client disconnect mid-conclude). +- The legacy single-blob flow is byte-identical on the wire before and after deployment. + +## References + +- [Forge blob removal RFC](./2026-07-forge-blob-removal.md) — blob lifecycle, `/blob/release`, `/blob/reject`, receipt-chain routing. +- [UCAN specification](https://github.com/ucan-wg/spec) — delegation, invocation, policy language. From c0acd4054054a6a2ce3c23335b7e1b647dd763b2 Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Sat, 5 Sep 2026 00:24:26 -0400 Subject: [PATCH 3/7] rfc: multi-blob add: delivery-based registration Agent drives both node verbs under placement delegations; the client completes at the accept round with no polling and one synchronous sprue exchange. Registration rides /ucan/conclude as open receipt delivery (node SHOULD conclude, agent SHOULD deliver; replica-transfer precedent), replacing the conclusion journal. Soundness additions from the adversarial pass: a delegation-expiry-derived registration horizon with matching node-side reclaim floor, cause-keyed and pending tombstones with per-(digest,space) serialization, batch-support advertisement gating placement, replica count in delegation metadata, and quota-debit rules. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 222 +++++++++++++++++++-------------- 1 file changed, 128 insertions(+), 94 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index a854714..aee861d 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -14,21 +14,23 @@ The clients that drive volume know every digest before the first network call. I This RFC proposes a batch upload protocol: -- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate nodes plus delegations that let the agent invoke `/blob/allocate` on those nodes directly. -- The agent drives allocation and upload itself: one container of `/blob/allocate` invocations per node, then PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land. Over-allocation and long-tail cancellation become placement policy rather than failure handling. -- `/ucan/conclude`, generalized to many receipts. For each blob that landed, the client concludes with the node-signed `/blob/allocate` receipt and the `/http/put` receipt. Sprue verifies the pair, drives `/blob/accept` at each winning node (one container per node), registers the blobs, and returns the accept receipts in the conclude response. +- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate nodes plus delegations that let the agent invoke `/blob/allocate` and `/blob/accept` on those nodes directly. +- The agent drives the entire node exchange itself: one container of `/blob/allocate` invocations per node, PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land, then one container of `/blob/accept` invocations per winning node. The accept receipts — location claim and PDP promise included — are in the agent's hands when the accept round completes. Nothing is polled and nothing is awaited from the upload service. +- Registration travels as receipt delivery over `/ucan/conclude`, which stays what it is today: an open conclusion mechanism that anyone holding a receipt may invoke. The storage node SHOULD conclude each accept it executes; the agent SHOULD deliver its collected receipts as well, batched. The upload service registers idempotently from whichever delivery arrives first. - `/blob/batch/extend` — replacement candidates when a blob exhausts its set. -Cost for an M-blob object: two client↔sprue round trips per batch, one agent↔node round trip per node plus the PUTs, and one sprue↔node round trip per winning node. No receipt poll: the conclude response carries the accept receipts. +Cost for an M-blob object: one synchronous round trip to the upload service, one allocate and one accept round trip per node plus the PUTs, and asynchronous receipt deliveries that never sit on the completion path. ## Goals -- Client↔upload-service round trips are independent of blob count: two per batch, plus one `/blob/batch/extend` exchange per repair round. -- No storage-node changes. Nodes keep serving `/blob/allocate` and `/blob/accept` exactly as today; only the allocate invoker changes. The protocol deploys against existing piri. -- Accept authority stays with the upload service. Accept is the liability-creating verb — the node mints and publishes a location claim, enqueues PDP, and the blob leaves the parked state that expiry can reclaim — so it remains invoked by the service, adjacent to the registration that accounts for it. -- The upload service drops out of the per-blob upload sequence. It mints capabilities without contacting a node, and touches nodes only at accept, after the client reports what landed. +- One synchronous client↔upload-service exchange per batch, independent of blob count, plus one `/blob/batch/extend` exchange per repair round. +- No polling and no conclusion wait: the agent holds every receipt it needs — accept, location claim, PDP promise — directly from the nodes, at the end of the node exchange. +- `/ucan/conclude` remains an intentionally open delivery mechanism; the upload service trusts what the delivered chain proves and nothing about who delivered it. +- Reporting duty sits with the party that bears the loss. An accepted blob nobody registers is unpaid bytes on the node, so the node concludes its own accepts; the agent's delivery is the second, independent path. +- Every acceptance has a protocol-visible registration horizon, derived from the delegation that authorized it, so acceptance, registration and reclaim can never disagree about whether a blob is live. +- The upload service drops off the upload path entirely after planning. It mints capabilities without contacting a node and never fans out to nodes at all. - Every node-side action remains traceable to a client-signed intent: the allocate's `cause` names the client's batch invocation, and the delegation policy binds it. -- Partial success is first-class. Blobs succeed and fail independently; conclusion is idempotent and resumable, so a straggler concludes later under the same batch. +- Partial success is first-class. Blobs succeed and fail independently; delivery is idempotent and repeatable, so stragglers converge under the same batch. - The receipt chain keeps today's shape — `add → allocate → put → accept` — so everything that walks receipts (replay, removal routing) is unchanged. ## Concepts @@ -43,12 +45,12 @@ Cost for an M-blob object: two client↔sprue round trips per batch, one agent | Space | A namespace (cryptographic key pair) that owns content. A bucket is a space. | | Ingot | An S3 facade typically co-located with a Forge Piri node. | -### The third delegation regime: re-delegated allocate +### The third delegation regime: re-delegated placement -The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: sprue re-delegates **`/blob/allocate` only** to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: +The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: sprue re-delegates `/blob/allocate` and `/blob/accept` to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: ```jsonc -// Policy on the /blob/allocate delegation (dag-json for readability) +// Policy on each placement delegation (dag-json for readability) [ ["==", ".space", "did:key:zSpace"], ["==", ".blob.digest", { "/": { "bytes": "EiCcvAfD..." } }], @@ -57,39 +59,38 @@ The existing upload protocol has two delegation regimes: space-rooted (the space ] ``` -The agent invokes allocate on the node under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can allocate only the announced digest, at the announced size, for the named space and cause, at the nodes sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. +The agent invokes the node verbs under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can act only on the announced digest, at the announced size, for the named space and cause, at the nodes sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. -Allocate is the safe verb to hand over: it is idempotent, keyed `(digest, space)`, and an allocation that never receives bytes is parked state the node's existing expiry reclaims. The worst an agent can do with an allocate delegation is create allocations sprue already decided to permit, which then expire. `/blob/accept` is deliberately **not** re-delegated: acceptance creates obligations that do not expire, so it stays a service-invoked verb, issued only after the client reports success and in the same step that registers the blob. +Two facts of each placement delegation do double duty later. Its metadata carries the blob's **replica count** (one, unless placement says otherwise), which bounds registration. Its **expiry** defines the registration horizon: the chain it roots can register only until that expiry plus a protocol grace period (see [Registration by receipt delivery](#registration-by-receipt-delivery)). -The baseline is one allocate delegation per blob per candidate node. For large batches sprue MAY compact to one delegation per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per placement, letting the agent retire its own parked pieces early instead of waiting for allocation expiry; this is safe because reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). +Handing over allocate is safe: it is idempotent, keyed `(digest, space)`, and an allocation that never receives bytes is parked state the node's existing expiry reclaims. Handing over accept is safe because registration follows the receipt rather than the invoker: acceptance creates the durable obligations — a published location claim, a PDP enqueue — and what keeps those from dangling is not who invoked accept but that the accept receipt reaches the upload service inside the horizon, which the node itself has the strongest incentive to ensure. -### Over-allocation and the winner report +The baseline is one delegation pair per blob per candidate node. For large batches sprue MAY compact to one pair per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per placement, letting the agent retire its own parked pieces early instead of waiting for allocation expiry; this is safe because reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). -Placement returns more candidates than a blob needs. The agent allocates at its candidates, PUTs to as many as it likes, and stops when enough land — the race is client policy, from "primary first, spare on failure" to "start all, cancel at first." A candidate whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and is an instant winner. +### Over-allocation and the race -The network learns the outcome from the client, in receipt form. For each blob that landed, the client holds two artifacts: +Placement returns more candidates than a blob needs. The agent allocates at its candidates, PUTs to as many as it likes, and stops when enough land — the race is client policy, from "primary first, spare on failure" to "start all, cancel at first." A candidate whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and is an instant winner. -- the **node-signed `/blob/allocate` receipt** — real evidence that the winning node precommitted to the piece, and the routing for everything that follows: the allocate invocation names the provider (its subject), the space, the digest and the cause, and its proof chain terminates in the delegation sprue itself minted; -- the **`/http/put` receipt** — the completion event, in the protocol's existing conclusion idiom. The put invocation is signed with a key derived from the blob digest, so it attests nothing cryptographically; it is the client's assertion that the bytes landed, and the protocol treats it as exactly that. The real check is the accept that follows: a false conclusion just produces a failed accept at the node. +The agent then invokes `/blob/accept` at winners, and MUST accept at no more than the blob's replica count of them: acceptance is the durable state, and registration will not pay for more. A retried accept that lands late anyway can still leave one extra acceptance; that converges rather than corrupts — the surplus node learns its status from its own delivery outcome and reclaims, readers who hit its stale location claim retry the registered node's claim, and the agent reconciles its index from the delivery outcomes. Losing candidates need no message at all: their parked allocations, and any bytes from PUTs cancelled after completion, expire node-side. ### The client-minted `/http/put` -Today sprue builds the `/http/put` invocation during `/blob/add`, deriving an ed25519 key from the blob digest and embedding it in the invocation metadata so the client can sign the receipt. In this protocol the agent performs the allocate, so the agent builds the put invocation itself — same derivation, same shape: +Today sprue builds the `/http/put` invocation during `/blob/add`, deriving an ed25519 key from the blob digest and embedding it in the invocation metadata so the client can sign the receipt. In this protocol the agent drives the node exchange, so the agent builds the put invocation itself — same derivation, same shape: - key: the multihash's 32-byte digest as an ed25519 seed; issuer = audience = subject = the derived `did:key`; - arguments: the blob and a `destination` await linking the agent's own `/blob/allocate` task; - receipt: self-signed with the derived key on PUT success — immediately, with no PUT, when the allocate receipt carried no upload address. -Nothing about the artifact's strength changes — anyone holding the digest could always mint it, which is the deliberate design — only where it is minted. The chain `batch add → allocate → put → accept` is exactly today's chain shape, and `/blob/accept`'s `_put` field awaits a real put task, as today. +Nothing about the artifact's strength changes — anyone holding the digest could always mint it, which is the deliberate design — only where it is minted. The agent's `/blob/accept` invocation links the put task in `_put`, so the chain `batch add → allocate → put → accept` is exactly today's chain shape. ### Batch lifecycle -1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the allocate delegations, and returns placements. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. -2. **Allocate and race** — one container of `/blob/allocate` invocations per candidate node; PUTs in parallel; cancel at enough. The agent mints a put receipt per winner. -3. **Conclude** — one `/ucan/conclude` carrying every winner's receipt pair. Sprue verifies the pairs, coalesces winners by provider, sends one `/blob/accept` container per winning node in parallel, registers each accepted blob with the batch-add task as its cause, and returns the accept receipts and per-blob outcomes in the response. -4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and concludes later under the same batch. +1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the placement delegations, and returns placements. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. +2. **Allocate, race, accept** — one container of `/blob/allocate` invocations per candidate node; PUTs in parallel; cancel at enough; one container of `/blob/accept` invocations per winning node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. +3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, registers idempotently, and answers with per-receipt outcomes. +4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and its receipts are delivered later under the same batch. ### Transport @@ -103,11 +104,11 @@ Batching travels as ucantone container contents: a single HTTP exchange carries * Audience: Sprue * Subject: The space -Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob with the minted delegations carried in the response container. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob with the minted delegations carried in the response container. Candidates come only from nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. -Idempotent: re-announcing a digest is safe — allocation is keyed `(digest, space)` on the node, and registration at conclusion tolerates the existing entry. +Idempotent: re-announcing a digest is safe — allocation is keyed `(digest, space)` on the node, and registration tolerates the existing entry. #### Arguments @@ -160,7 +161,7 @@ e.g. type Placement struct { digest Bytes # multihash of the blob provider String # DID of the candidate storage node - proofs [Link] # delegations in the response container: /blob/allocate, optionally /blob/reject + proofs [Link] # delegations in the response container: /blob/allocate, /blob/accept, optionally /blob/reject } type BatchAddOK struct { @@ -190,32 +191,28 @@ type BatchAddOK struct { ### `/ucan/conclude`, generalized -* Issuer: Agent +* Issuer: any principal holding the receipts — typically the storage node or the agent * Audience: Sprue -* Subject: The space +* Subject: the issuer (self-issued, as today) -The existing conclusion command, extended to carry many receipts. The arguments gain an optional plural field; a server implementing this RFC MUST accept both forms, and the legacy single-receipt flow is untouched. +The existing conclusion command, extended to carry many receipts. Conclude is deliberately an **open receipt-delivery mechanism**: it carries no authority of its own, anyone may invoke it, and the upload service acts only on what the delivered artifacts prove. Piri already uses it outbound this way, delivering its replica-transfer receipts to the upload service. The arguments gain an optional plural field — exactly one of `receipt` or `receipts` MUST be present — and a server implementing this RFC MUST accept both forms; the legacy single-receipt flow is untouched. -The multi-receipt form also tightens authorization, deliberately. Today's conclude is effectively self-authorized — the client issues it with itself as subject, and the handler checks only that the named receipt corresponds to a stored invocation. The multi form MUST carry the space as subject, proven by the agent's space-rooted chain, and registration MUST bind to the space the placement delegation pins, which MUST equal the invocation subject. This closes conclusion of another space's uploads by anyone who learns a task CID. +For a delivered `/blob/accept` receipt to register a blob, the request container MUST make the full chain resolvable: the accept invocation, its node-signed receipt, the put invocation its `_put` field awaits, and the `/blob/allocate` invocation that put's `destination` awaits. Deliverers SHOULD include the allocate receipt as well, so sprue persists a complete chain for receipt-chain consumers. -For each concluded `/http/put` receipt, the request container MUST make the winner's pair resolvable: the put invocation, and — via its `destination` await — the `/blob/allocate` invocation and its node-signed receipt. The handler resolves the pair from the request container; resolution from sprue's agent store remains for the legacy single form, whose artifacts sprue itself wrote. +Sprue dispatches each delivered receipt whose `Ran` command has a registered conclusion handler; other artifacts in the container are context. The new `/blob/accept` conclusion handler, per delivery: -Sprue dispatches each receipt whose `Ran` command has a registered conclusion handler; other receipts in the container are context. The `/http/put` handler, per conclusion: +1. verifies the accept receipt is signed by the provider DID that is the accept invocation's subject, with `Ran` matching and a success result; +2. verifies the accept invocation's proof chain terminates in a **placement delegation sprue itself signed**, whose subject is the provider and whose policy pins the space, digest, size and cause. Sprue verifying its own signature is the placement check, and it requires no stored state; +3. enforces the registration horizon: a chain whose placement delegation expired more than the protocol grace period ago MUST be refused with an `Expired` outcome. Within the horizon, freshness is not otherwise rechecked — the node-signed receipt attests that the chain, expiry included, validated at execution time; +4. enforces the registration bound: at most the replica count the placement delegation's metadata names — one, absent the field — is registered per `(digest, space)`. The first delivered acceptance wins; later acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; +5. checks the tombstones (below): a chain matching one returns a `Removed` outcome instead of registering; +6. registers the blob against the space named in the verified chain, with the batch-add task as `cause`, debits the space's quota, and persists the delivered invocations and receipts to the agent store. -1. follows `destination` to the allocate invocation and receipt; -2. verifies the allocate invocation's proof chain terminates in a **placement delegation sprue itself signed**, whose audience is the concluding agent and whose policy pins the provider, space, digest, size and cause — and that the pinned space equals the conclude subject. Sprue verifying its own signature is the placement check, and it requires no stored state. Verification checks signature and policy bindings, **not** freshness: the node-signed allocate receipt attests that the full chain, expiry included, validated at execution time. Provider registration and liveness are checked at accept time; -3. verifies the node's signature on the allocate receipt; -4. enforces the placement bound: at most the placement policy's replica count of providers — one, unless placement says otherwise — is accepted per `(digest, space)`. The first verified pair wins; surplus pairs return a `SurplusWinner` outcome with no accept, and registration records the accepted provider. This bound, not delegation revocation, is what caps winners — an extend's exclusions revoke nothing; -5. invokes `/blob/accept` at the provider — the verification the protocol actually relies on: the node confirms it holds the bytes. Within one conclude request sprue SHOULD coalesce accepts by provider into one container per node, sent in parallel, each under a mandatory per-node deadline so no stalled node delays the response; -6. on accept success, registers the blob against the space with the batch-add task as `cause`, and persists the relayed invocations and receipts to its agent store. +Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement delegation was issued before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). Both checks read only the delivered delegation, so verification stays stateless. -The multi-receipt form returns per-conclusion outcomes, with the accept receipts in the response container — this is what retires the receipt poll. A conclusion failing any check lands in the outcome list with a named reason and blocks nothing else. +Delivery is idempotent and unordered: the node's conclusion and the agent's conclusion of the same accept converge on one registration, and re-delivering is always safe. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. -A batch never has to conclude whole. Conclusion registers whatever verifies; a later conclude under the same batch picks up stragglers. Registration and acceptance tolerate repeats, so overlapping or repeated conclusions converge. Clients SHOULD conclude within the delegation lifetime: node-side allocation expiry can reap unconcluded bytes, and the node's deadline is not visible to sprue. - -Idempotent: concluding an already-accepted blob MUST succeed. - -Implementation note: the container accessor for receipts is a linear scan. Matching M conclusions to their container artifacts naively is O(M²); index the container's contents by link and by `Ran()` once before the loop. +Implementation note: the container accessor for receipts is a linear scan. Matching M deliveries to their container artifacts naively is O(M²); index the container's contents by link and by `Ran()` once before the loop. #### Arguments @@ -224,7 +221,7 @@ Implementation note: the container accessor for receipts is a linear scan. Match ```ipldsch type ConcludeArguments struct { receipt optional Link # legacy single form - receipts optional [Link] # the /http/put receipts being concluded + receipts optional [Link] # the receipts being delivered } ``` @@ -242,18 +239,18 @@ type ConcludeArguments struct { e.g. ```jsonc -// Encoded as dag-json for readability; the referenced put receipts, -// put invocations, allocate invocations and allocate receipts travel +// Encoded as dag-json for readability; the referenced accept receipts, +// accept invocations, put invocations and allocate invocations travel // in the same container { - "iss": "did:key:agent", + "iss": "did:key:zPiriNode", "aud": "did:web:up.forge.example.com", - "sub": "did:key:space", + "sub": "did:key:zPiriNode", "cmd": "/ucan/conclude", "args": { "receipts": [ - { "/": "bafyreia...putrcpt1" }, - { "/": "bafyreib...putrcpt2" } + { "/": "bafyreia...acceptrcpt1" }, + { "/": "bafyreib...acceptrcpt2" } ] } } @@ -267,11 +264,10 @@ The legacy single form keeps its current result. The multi form returns: ```ipldsch type ConcludeOutcome struct { - receipt Link # the concluded /http/put receipt + receipt Link # the delivered receipt registered Bool - accept optional Link # the node-signed /blob/accept receipt, in the response container - reason optional String # named error, e.g. InvalidAllocateReceipt, UnknownPlacement, - # SurplusWinner, AcceptRefused, AcceptTimeout + reason optional String # named outcome, e.g. AlreadyRegistered, Surplus, Removed, + # Expired, InvalidReceipt, UnknownPlacement } type BatchConcludeOK struct { @@ -284,10 +280,9 @@ type BatchConcludeOK struct { ```go type ConcludeOutcome struct { - Receipt cid.Cid `cborgen:"receipt"` - Registered bool `cborgen:"registered"` - Accept *cid.Cid `cborgen:"accept,omitempty"` - Reason string `cborgen:"reason,omitempty"` + Receipt cid.Cid `cborgen:"receipt"` + Registered bool `cborgen:"registered"` + Reason string `cborgen:"reason,omitempty"` } type BatchConcludeOK struct { @@ -302,7 +297,7 @@ type BatchConcludeOK struct { * Audience: Sprue * Subject: The space -Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh candidates with fresh delegations — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch concludes under one cause. +Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh candidates with fresh delegations, opening fresh quota holds and a fresh registration horizon for the re-placed blobs — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch delivers under one cause. Idempotent: extending a digest that has since been accepted or registered simply returns no placement for it, listed in `registered`. @@ -338,57 +333,91 @@ type BatchExtendArguments struct { ``` +e.g. + +```jsonc +// Encoded as dag-json for readability +{ + "iss": "did:key:agent", + "aud": "did:web:up.forge.example.com", + "sub": "did:key:space", + "cmd": "/blob/batch/extend", + "args": { + "cause": { "/": "bafy...batchaddtask" }, + "blobs": [ + { + "digest": { "/": { "bytes": "EiCcvAfD..." } }, + "exclude": ["did:key:zFailedNode"] + } + ] + } +} +``` + #### Result `BatchAddOK`, as for [`/blob/batch/add`](#blobbatchadd). -### The node legs: `/blob/allocate` and `/blob/accept`, unchanged +### The node legs: `/blob/allocate` and `/blob/accept`, unchanged verbs + +* Issuer: Agent +* Audience: Piri +* Subject: The provider The storage-node verbs, their argument types and their results are exactly today's. What changes: -- **`/blob/allocate`** is invoked by the agent under the re-delegation, with `cause` = the `/blob/batch/add` task CID (bound by the delegation policy), so every allocation row a node writes names the client invocation that motivated it. -- **`/blob/accept`** is invoked by sprue under its registration delegation, as today, with `_put` awaiting the client-minted `/http/put` task from the concluded pair. +- Both are invoked by the agent under the placement delegations, with `cause` = the `/blob/batch/add` task CID (bound by policy), so every allocation and acceptance a node writes names the client invocation that motivated it. +- `/blob/accept`'s `_put` field awaits the client-minted `/http/put` task, preserving the chain shape. +- A node that serves batch placements advertises **batch support** at registration, which commits it to two behaviors. It SHOULD deliver each accept it executes to the upload service via `/ucan/conclude` (see [Registration by receipt delivery](#registration-by-receipt-delivery)), retrying until acknowledged. And it reclaims acceptances that remain unregistered past its reclaim deadline — learned from its delivery outcomes (`Surplus`, `Removed`, `Expired`) or from sustained delivery failure — where the deadline MUST be at least the placement delegation's expiry plus the protocol grace period, so reclaim can never race a chain that could still register. Beyond that floor the deadline is node policy. -## Registration durability +## Registration by receipt delivery -Accept at the node and registration at sprue are two steps, and the gap between them is where accepted-but-unregistered state — a node holding non-expiring, PDP-enqueued bytes no space is registered for — would otherwise leak: an accept that times out after the node committed, a sprue crash between accept and registration, a client that disconnects mid-conclude. The protocol closes the gap by making completion sprue's obligation rather than the request's: +Acceptance happens at the node; registration happens at sprue; and the bridge between them is receipt delivery. Delivery removes the coordination state the upload service would otherwise need — no journal, no accept fan-out, no timeout vocabulary — because the evidence sits with the parties who can act on it: -- Sprue MUST journal each verified conclusion durably **before** invoking accept, and MUST drive the journaled pair through accept and registration to completion regardless of the request — the handler runs to completion on client disconnect, and a reconciliation sweep re-drives journaled pairs orphaned by a crash or timeout. -- The outcome vocabulary keeps the two failure meanings apart. `AcceptRefused` means the node rejected the accept: the blob is not stored there, and re-placing via `/blob/batch/extend` is the right response. `AcceptTimeout` means the outcome is unknown and sprue owns reconciliation: the client MUST NOT re-place the blob on this signal — re-concluding later is safe and returns the settled outcome. +- **The node holds the receipt it minted and bears the loss if registration never happens** — an accepted blob nobody registers is bytes it stores and proves unpaid. So the node SHOULD conclude every accept it executes, retrying until acknowledged. +- **The agent holds the same receipts** and SHOULD deliver them batched — typically immediately, as its confirmation that registration is underway. Either delivery alone suffices; both together make loss require two independent failures. +- **Sprue does only local work per delivery**: verify the chain, apply the horizon, the bound and the tombstones, register, persist. Registration is idempotent, so redundant and repeated deliveries converge, and a delivery can be replayed at any time by anyone still holding the receipts. -With the journal, a verified conclusion either completes or is retried by sprue itself; no client behavior, node stall, or crash strands an accept unregistered beyond the sweep interval. +The **registration horizon** is what keeps the three parties' clocks consistent. A chain registers only until its placement delegation's expiry plus a protocol grace period; the node's reclaim deadline starts at that same boundary. So a chain that can still register names bytes the node still holds, a reclaimed acceptance names a chain that can no longer register, and a months-old receipt delivered by anyone is refused rather than resurrected as a billed ghost. The failure this makes visible instead of silent: a delivery that misses the horizon returns `Expired`, the node reclaims, and the client re-uploads under a fresh batch — rather than sprue registering bytes no node holds. + +The residual window is bounded: a blob is accepted-but-unregistered until the first delivery lands, and if every holder crashes before delivering, until one recovers or the horizon closes it out. + +The client's completion does not wait for any of this. Resolvability comes from accept itself (the node publishes its location claim synchronously, as today), so the S3 200 returns on accept receipts; registration — billing, retain seeding, repair and removal routing — converges with delivery, typically within the agent's own immediate conclude. ## Failure handling and idempotency - A failed or stalled PUT races another candidate; no sprue call. An exhausted candidate set costs one `/blob/batch/extend`. -- An `AcceptRefused` outcome sends the blob back through extend and a later conclude; an `AcceptTimeout` outcome is sprue's to reconcile (see [Registration durability](#registration-durability)). +- A refused accept is visible to the agent directly in the node's response; the blob goes back through the race or through extend. +- An `Expired` outcome means the batch's registration window lapsed undelivered; the blob re-uploads under a fresh batch. +- The agent SHOULD reconcile its index against its delivery outcomes: a `Surplus` outcome for the receipt it indexed means a different winner registered, and the registered node's location claim is the durable one. - Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. -- Every capability in this RFC is idempotent: re-adding, re-concluding and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". +- Every capability in this RFC is idempotent: re-adding, re-delivering and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". ## Security considerations -- **Attenuation.** The allocate delegation's policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can produce exactly the parked allocations sprue decided to permit, nowhere else, and only until expiry. -- **Accept authority never moves.** Accept — the step that mints a published location claim, enqueues PDP, and exits the expiry-reclaimable state — is invoked only by sprue, only after a client conclusion, and its completion is journaled (see [Registration durability](#registration-durability)), so no client can manufacture an accepted-but-unregistered blob. +- **Attenuation.** The placement delegations' policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can act only on the placements sprue decided to permit, nowhere else, and only until expiry. +- **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt whose invocation chain terminates in sprue's own placement delegation, inside the horizon. The conclude issuer is irrelevant by design — a leaked receipt chain delivered by a stranger causes at most a correct, in-horizon registration of the space the chain names. - **No new agent trust.** The agent gains no authority it could not already exercise through `/blob/add`; sprue's placement discretion is unchanged, exercised at plan time instead of invocation time. -- **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and the protocol treats it as one. The node-signed allocate receipt is the signed artifact, and accept succeeding at the node is the verification the protocol relies on. -- **Replay.** Delegations expire; invocations carry nonces; allocation is keyed `(digest, space)`, so a replayed allocate reproduces the same row rather than a new liability; re-conclusion and re-registration converge, and the conclude subject binding stops cross-space conclusion of a leaked pair. -- **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint delegations for M blobs, and one conclude makes it verify and fan out accepts for M winners. Sprue SHOULD enforce configurable caps on blobs per batch and conclusions per conclude. Quota SHOULD be taken as a hold with a TTL equal to the delegation expiry, released on expiry and reconciled at registration — an outright debit leaks on abandoned batches, and a bare check over-commits across parallel batches. +- **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that over-accepts gains nothing durable and costs the nodes only until reclaim. +- **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and nothing rests on it; the node-signed accept receipt is the artifact registration relies on. +- **Replay.** Delegations expire; invocations carry nonces; allocation and acceptance are keyed `(digest, space)`; delivery, registration and reclaim all converge under repeats; the horizon refuses stale chains outright, and the tombstones stop a replayed delivery from resurrecting a removed registration in either race direction. +- **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint delegations for M blobs, and one conclude makes it verify M chains. Sprue SHOULD enforce configurable caps on blobs per batch and receipts per conclude. Quota SHOULD be taken as a hold with a TTL equal to the delegation expiry, released on expiry and reconciled at registration; registration inside the horizon debits unconditionally, so the over-commit exposure is bounded to chains delivered in the grace period after their hold released — an outright debit at add would leak on abandoned batches, and a bare check would over-commit across parallel batches. ## Limits -The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-candidate delegation baseline puts one or two signed delegations per candidate, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand candidates. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-candidate delegation baseline puts two or three signed delegations per candidate, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand candidates. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. ## Compatibility -- **Storage nodes**: no change. The provider → sprue → agent allocate chain validates under existing subject-plus-policy checking. -- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, race, conclude, repeat. -- **`/ucan/conclude`**: the plural field is additive; existing single-receipt clients and the legacy flow are untouched. -- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept` — has today's shape, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md#receipt-chain-routing)'s receipt-chain routing work unchanged for concluded blobs. Never-concluded blobs differ: sprue never saw their allocates, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry, which reconciles parked pieces with no message at all; the optional `/blob/reject` delegation in the placement proofs gives an agent early reclaim directly against the node. -- **Read-on-write**: preserved. The node still publishes its location claim synchronously inside accept, and registration completes inside the conclude request, so a successful conclusion still implies the blob is resolvable. Index and upload registration (`/index/add`, `/upload/add`) follow conclusion exactly as they follow the accept receipt today, and are out of scope here. +- **Storage nodes**: the verbs, argument types and results are unchanged, and the provider → sprue → agent chain validates under existing subject-plus-policy checking. Outbound conclusion has an existing precedent — piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. +- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, race, accept, deliver, repeat. +- **`/ucan/conclude`**: the plural field is additive, and the `/blob/accept` conclusion handler is a new entry in the existing dispatch map; existing single-receipt clients and the legacy `/http/put` handler are untouched. +- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept` — has today's shape, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md)'s receipt-chain routing work unchanged for delivered blobs. Undelivered blobs differ: sprue has not seen their chains, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry and the reclaim deadline; the optional `/blob/reject` delegation gives an agent early reclaim directly against the node. +- **Read-on-write**: resolvability is preserved at its source — the node publishes its location claim synchronously inside accept, before the agent ever holds the receipt. Registration-dependent behavior converges with delivery. Index and upload registration (`/index/add`, `/upload/add`) follow the accept receipts exactly as today, and are out of scope here. ## Future work -- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it, and the delegation regime is the same. +- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it — the delegation regime and the delivery model are the same. ## Example flow @@ -397,30 +426,35 @@ An object of M blobs raced across N candidate nodes: | # | Leg | What moves | |---|-----|------------| | 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | -| 2 | sprue → agent | `BatchAddOK`: candidates per blob, allocate delegations in the container | +| 2 | sprue → agent | `BatchAddOK`: candidates per blob, placement delegations in the container | | 3 | agent → each node, parallel | one container batching that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — warm, instant winner) | | 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; mint a put receipt per winner | -| 5 | agent → sprue | one `/ucan/conclude` carrying every winner's allocate + put receipt pair | -| 6 | sprue → each winning node, parallel | one container batching that node's `/blob/accept` invocations | -| 7 | sprue → agent | `BatchConcludeOK`: per-blob outcomes, accept receipts in the container; the S3 200 follows | +| 5 | agent → each winning node, parallel | one container batching that node's `/blob/accept` invocations; receipts carry location claims and PDP promises — **the client is done** | +| 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | +| 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response reports per-receipt registration outcomes | + +A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs that failed everywhere, repeats legs 3–5 against the fresh candidates, and the late receipts ride the same delivery path under the original cause. -Round trips: two to sprue per batch, one per node from the agent, one per winning node from sprue — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 42 control exchanges, all parallel across nodes, instead of ~500, serial per blob. Byte PUTs are excluded from both counts. +Synchronous round trips: one to sprue per batch, two per node — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 41 synchronous control exchanges, all parallel across nodes, plus at most 21 asynchronous receipt deliveries off the completion path — instead of ~500, serial per blob. Byte PUTs are excluded from both counts. ## Open questions +- The value of the registration grace period, and of the delegation expiry it extends (the delegation lifetime is also the upload window, so it wants to be generous — hours, not seconds). +- The mechanics of the batch-support advertisement at registration: a capability proven in the registration container, alongside the four commands required today, or a flag on the provider record. +- Whether the agent awaits its conclude response before the S3 200 (stronger registration signal) or fires it after (lower latency); this RFC leans to the latter since resolvability does not depend on it. - Default and maximum sizes for the batch, conclude and candidate-set caps. - Whether `/blob/reject` delegations ship in placement proofs by default or on request. -- The per-node accept deadline and the reconciliation sweep interval. - Whether the per-provider `or`-compacted delegation form ships in v1 or waits for a batch size that needs it. ## Evaluation criteria -- Control exchanges for an M-blob object drop from ~5M to 2 + 2N, measured on ingot's sharded-object upload. -- End-to-end upload latency for a multi-blob object improves by at least the removed poll and serialization overhead, measured against the current guppy/ingot path. -- No growth in stranded state: accepted-but-unregistered blobs stay at zero under fault injection (accept timeout, sprue restart mid-conclude, client disconnect mid-conclude). +- Synchronous control exchanges for an M-blob object drop from ~5M to 1 + 2N, measured on ingot's sharded-object upload, with zero polling. +- End-to-end upload latency for a multi-blob object improves by at least the removed poll, conclusion wait and serialization overhead, measured against the current guppy/ingot path. +- No growth in stranded state: accepted-but-unregistered blobs converge to zero — registered, or reclaimed with a visible `Expired`/`Surplus`/`Removed` outcome — under fault injection: node crash before conclude, agent crash before conclude, sprue restart, `/blob/remove` racing a late delivery in both directions, and delivery after the horizon. - The legacy single-blob flow is byte-identical on the wire before and after deployment. ## References - [Forge blob removal RFC](./2026-07-forge-blob-removal.md) — blob lifecycle, `/blob/release`, `/blob/reject`, receipt-chain routing. +- [Blob protocol specification](https://github.com/fil-forge/ucan-protocol-specs/blob/main/blob.md) — the current single-blob add protocol this RFC batches. - [UCAN specification](https://github.com/ucan-wg/spec) — delegation, invocation, policy language. From f63c720d8c24637710891858da2bc9d8d4e5a8d0 Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Fri, 4 Sep 2026 22:29:08 -0700 Subject: [PATCH 4/7] rfc: multi-blob add: /blob/register terminal chain link Registration becomes a chain-visible virtual task in the /http/put tradition: sprue mints it per winning acceptance inside the /blob/accept conclusion handler, no-nonce so concurrent deliveries converge on one receipt. ConcludeOutcome carries a registration link instead of a boolean; Surplus outcomes link the registered acceptances; removal retires the register tasks via the cause-keyed tombstone. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 85 ++++++++++++++++++++++++++-------- 1 file changed, 65 insertions(+), 20 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index aee861d..6ca1ed9 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -31,7 +31,7 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one - The upload service drops off the upload path entirely after planning. It mints capabilities without contacting a node and never fans out to nodes at all. - Every node-side action remains traceable to a client-signed intent: the allocate's `cause` names the client's batch invocation, and the delegation policy binds it. - Partial success is first-class. Blobs succeed and fail independently; delivery is idempotent and repeatable, so stragglers converge under the same batch. -- The receipt chain keeps today's shape — `add → allocate → put → accept` — so everything that walks receipts (replay, removal routing) is unchanged. +- The receipt chain keeps today's shape — `add → allocate → put → accept` — so everything that walks receipts (replay, removal routing) is unchanged, and gains a sprue-signed terminal link: registration itself becomes a chain-visible `/blob/register` receipt. ## Concepts @@ -89,7 +89,7 @@ Nothing about the artifact's strength changes — anyone holding the digest coul 1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the placement delegations, and returns placements. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. 2. **Allocate, race, accept** — one container of `/blob/allocate` invocations per candidate node; PUTs in parallel; cancel at enough; one container of `/blob/accept` invocations per winning node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. -3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, registers idempotently, and answers with per-receipt outcomes. +3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, mints the `/blob/register` receipt idempotently, and answers with per-receipt outcomes linking it. 4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and its receipts are delivered later under the same batch. ### Transport @@ -204,13 +204,13 @@ Sprue dispatches each delivered receipt whose `Ran` command has a registered con 1. verifies the accept receipt is signed by the provider DID that is the accept invocation's subject, with `Ran` matching and a success result; 2. verifies the accept invocation's proof chain terminates in a **placement delegation sprue itself signed**, whose subject is the provider and whose policy pins the space, digest, size and cause. Sprue verifying its own signature is the placement check, and it requires no stored state; 3. enforces the registration horizon: a chain whose placement delegation expired more than the protocol grace period ago MUST be refused with an `Expired` outcome. Within the horizon, freshness is not otherwise rechecked — the node-signed receipt attests that the chain, expiry included, validated at execution time; -4. enforces the registration bound: at most the replica count the placement delegation's metadata names — one, absent the field — is registered per `(digest, space)`. The first delivered acceptance wins; later acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; +4. enforces the registration bound: at most the replica count the placement delegation's metadata names — one, absent the field — is registered per `(digest, space)`. The earliest delivered acceptances win, up to the bound; further acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; 5. checks the tombstones (below): a chain matching one returns a `Removed` outcome instead of registering; -6. registers the blob against the space named in the verified chain, with the batch-add task as `cause`, debits the space's quota, and persists the delivered invocations and receipts to the agent store. +6. registers the blob by minting one [`/blob/register`](#blobregister) task per winning acceptance and issuing its receipt, debits the space's quota, and persists the delivered chain and the registration alongside it. -Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement delegation was issued before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). Both checks read only the delivered delegation, so verification stays stateless. +Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` tasks and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement delegation was issued before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). Both checks read only the delivered delegation, so verification stays stateless. -Delivery is idempotent and unordered: the node's conclusion and the agent's conclusion of the same accept converge on one registration, and re-delivering is always safe. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. +Delivery is idempotent and unordered: the `/blob/register` task is built with no nonce, so the node's conclusion and the agent's conclusion of the same accept converge on the identical task CID and one receipt, and a repeat delivery returns the same registration link. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else, and a `Surplus` outcome still carries the blob's registration links, so the deliverer learns the registered providers directly. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. Implementation note: the container accessor for receipts is a linear scan. Matching M deliveries to their container artifacts naively is O(M²); index the container's contents by link and by `Ran()` once before the loop. @@ -264,10 +264,10 @@ The legacy single form keeps its current result. The multi form returns: ```ipldsch type ConcludeOutcome struct { - receipt Link # the delivered receipt - registered Bool - reason optional String # named outcome, e.g. AlreadyRegistered, Surplus, Removed, - # Expired, InvalidReceipt, UnknownPlacement + receipt Link # the delivered receipt + registration optional Link # the /blob/register receipt, in the response container + reason optional String # named outcome when this delivery registered nothing, + # e.g. Surplus, Removed, Expired, InvalidReceipt, UnknownPlacement } type BatchConcludeOK struct { @@ -280,9 +280,9 @@ type BatchConcludeOK struct { ```go type ConcludeOutcome struct { - Receipt cid.Cid `cborgen:"receipt"` - Registered bool `cborgen:"registered"` - Reason string `cborgen:"reason,omitempty"` + Receipt cid.Cid `cborgen:"receipt"` + Registration *cid.Cid `cborgen:"registration,omitempty"` + Reason string `cborgen:"reason,omitempty"` } type BatchConcludeOK struct { @@ -291,6 +291,50 @@ type BatchConcludeOK struct { ``` +### `/blob/register` + +* Issuer: Sprue +* Audience: Sprue +* Subject: Sprue + +The terminal link of the upload chain, and a virtual task in the tradition of `/http/put`: nothing serves it. Sprue mints it inside the `/blob/accept` conclusion handler when a delivery verifies, issues the receipt itself, and persists both with the delivered chain. The receipt is the canonical, sprue-signed record that a blob is registered — the artifact billing, index reconciliation and disputes start from — and the registration time is the receipt's issue time. + +The task is built with no nonce, so its CID is deterministic over its arguments: every delivery of the same acceptance names the identical task, and exactly one receipt exists per registered acceptance no matter who delivered first or how many times. Removing a registered blob retires its register task per acceptance; the cause-keyed tombstone records them. + +Idempotent: a delivery that finds the registration already minted returns the existing receipt's link in its outcome. + +#### Arguments + +**IPLD schema** + +```ipldsch +type RegisterArguments struct { + space String # DID of the space the blob is registered to + blob Blob # digest and size + provider String # DID of the storage node holding the registered acceptance + cause Link # the /blob/batch/add task + accept Link # the /blob/accept task whose delivery triggered registration +} +``` + +
+Go syntax + +```go +type RegisterArguments struct { + Space did.DID `cborgen:"space"` + Blob blob.Blob `cborgen:"blob"` + Provider did.DID `cborgen:"provider"` + Cause cid.Cid `cborgen:"cause"` + Accept cid.Cid `cborgen:"accept"` +} +``` +
+ +#### Result + +Successful registration returns a unit result (`{}`). The receipt's substance is its signature and issue time. + ### `/blob/batch/extend` * Issuer: Agent @@ -376,27 +420,27 @@ Acceptance happens at the node; registration happens at sprue; and the bridge be - **The node holds the receipt it minted and bears the loss if registration never happens** — an accepted blob nobody registers is bytes it stores and proves unpaid. So the node SHOULD conclude every accept it executes, retrying until acknowledged. - **The agent holds the same receipts** and SHOULD deliver them batched — typically immediately, as its confirmation that registration is underway. Either delivery alone suffices; both together make loss require two independent failures. -- **Sprue does only local work per delivery**: verify the chain, apply the horizon, the bound and the tombstones, register, persist. Registration is idempotent, so redundant and repeated deliveries converge, and a delivery can be replayed at any time by anyone still holding the receipts. +- **Sprue does only local work per delivery**: verify the chain, apply the horizon, the bound and the tombstones, mint the registration, persist. Registration is idempotent, so redundant and repeated deliveries converge on one `/blob/register` receipt, and a delivery can be replayed at any time by anyone still holding the receipts. The **registration horizon** is what keeps the three parties' clocks consistent. A chain registers only until its placement delegation's expiry plus a protocol grace period; the node's reclaim deadline starts at that same boundary. So a chain that can still register names bytes the node still holds, a reclaimed acceptance names a chain that can no longer register, and a months-old receipt delivered by anyone is refused rather than resurrected as a billed ghost. The failure this makes visible instead of silent: a delivery that misses the horizon returns `Expired`, the node reclaims, and the client re-uploads under a fresh batch — rather than sprue registering bytes no node holds. The residual window is bounded: a blob is accepted-but-unregistered until the first delivery lands, and if every holder crashes before delivering, until one recovers or the horizon closes it out. -The client's completion does not wait for any of this. Resolvability comes from accept itself (the node publishes its location claim synchronously, as today), so the S3 200 returns on accept receipts; registration — billing, retain seeding, repair and removal routing — converges with delivery, typically within the agent's own immediate conclude. +The client's completion does not wait for any of this. Resolvability comes from accept itself (the node publishes its location claim synchronously, as today), so the S3 200 returns on accept receipts; registration — billing, retain seeding, repair and removal routing — converges with delivery, typically within the agent's own immediate conclude. A client that wants a stronger signal than the 200 holds one artifact: the `/blob/register` receipt linked from its delivery outcome, verifiable offline and presentable to anyone. ## Failure handling and idempotency - A failed or stalled PUT races another candidate; no sprue call. An exhausted candidate set costs one `/blob/batch/extend`. - A refused accept is visible to the agent directly in the node's response; the blob goes back through the race or through extend. - An `Expired` outcome means the batch's registration window lapsed undelivered; the blob re-uploads under a fresh batch. -- The agent SHOULD reconcile its index against its delivery outcomes: a `Surplus` outcome for the receipt it indexed means a different winner registered, and the registered node's location claim is the durable one. +- The agent SHOULD reconcile its index against the `/blob/register` receipt linked from its delivery outcomes: it names the registered provider, whose location claim is the durable one. - Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. - Every capability in this RFC is idempotent: re-adding, re-delivering and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". ## Security considerations - **Attenuation.** The placement delegations' policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can act only on the placements sprue decided to permit, nowhere else, and only until expiry. -- **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt whose invocation chain terminates in sprue's own placement delegation, inside the horizon. The conclude issuer is irrelevant by design — a leaked receipt chain delivered by a stranger causes at most a correct, in-horizon registration of the space the chain names. +- **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt whose invocation chain terminates in sprue's own placement delegation, inside the horizon. The conclude issuer is irrelevant by design — a leaked receipt chain delivered by a stranger causes at most a correct, in-horizon registration of the space the chain names. Registration produces evidence in turn: the sprue-signed `/blob/register` receipt is the record billing and disputes start from. - **No new agent trust.** The agent gains no authority it could not already exercise through `/blob/add`; sprue's placement discretion is unchanged, exercised at plan time instead of invocation time. - **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that over-accepts gains nothing durable and costs the nodes only until reclaim. - **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and nothing rests on it; the node-signed accept receipt is the artifact registration relies on. @@ -412,7 +456,7 @@ The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The p - **Storage nodes**: the verbs, argument types and results are unchanged, and the provider → sprue → agent chain validates under existing subject-plus-policy checking. Outbound conclusion has an existing precedent — piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. - **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, race, accept, deliver, repeat. - **`/ucan/conclude`**: the plural field is additive, and the `/blob/accept` conclusion handler is a new entry in the existing dispatch map; existing single-receipt clients and the legacy `/http/put` handler are untouched. -- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept` — has today's shape, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md)'s receipt-chain routing work unchanged for delivered blobs. Undelivered blobs differ: sprue has not seen their chains, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry and the reclaim deadline; the optional `/blob/reject` delegation gives an agent early reclaim directly against the node. +- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept`, closed by a sprue-signed `register` — has today's shape in its prefix, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md)'s receipt-chain routing work unchanged for delivered blobs; consumers MAY also start from the registration receipt, which links the accept task directly. Undelivered blobs differ: sprue has not seen their chains, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry and the reclaim deadline; the optional `/blob/reject` delegation gives an agent early reclaim directly against the node. - **Read-on-write**: resolvability is preserved at its source — the node publishes its location claim synchronously inside accept, before the agent ever holds the receipt. Registration-dependent behavior converges with delivery. Index and upload registration (`/index/add`, `/upload/add`) follow the accept receipts exactly as today, and are out of scope here. ## Future work @@ -431,7 +475,7 @@ An object of M blobs raced across N candidate nodes: | 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; mint a put receipt per winner | | 5 | agent → each winning node, parallel | one container batching that node's `/blob/accept` invocations; receipts carry location claims and PDP promises — **the client is done** | | 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | -| 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response reports per-receipt registration outcomes | +| 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response links a sprue-signed `/blob/register` receipt per registered blob | A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs that failed everywhere, repeats legs 3–5 against the fresh candidates, and the late receipts ride the same delivery path under the original cause. @@ -441,7 +485,7 @@ Synchronous round trips: one to sprue per batch, two per node — against three - The value of the registration grace period, and of the delegation expiry it extends (the delegation lifetime is also the upload window, so it wants to be generous — hours, not seconds). - The mechanics of the batch-support advertisement at registration: a capability proven in the registration container, alongside the four commands required today, or a flag on the provider record. -- Whether the agent awaits its conclude response before the S3 200 (stronger registration signal) or fires it after (lower latency); this RFC leans to the latter since resolvability does not depend on it. +- Whether the agent awaits its conclude response — and with it the `/blob/register` receipt — before the S3 200, or fires it after; this RFC leans to the latter since resolvability does not depend on it. - Default and maximum sizes for the batch, conclude and candidate-set caps. - Whether `/blob/reject` delegations ship in placement proofs by default or on request. - Whether the per-provider `or`-compacted delegation form ships in v1 or waits for a batch size that needs it. @@ -451,6 +495,7 @@ Synchronous round trips: one to sprue per batch, two per node — against three - Synchronous control exchanges for an M-blob object drop from ~5M to 1 + 2N, measured on ingot's sharded-object upload, with zero polling. - End-to-end upload latency for a multi-blob object improves by at least the removed poll, conclusion wait and serialization overhead, measured against the current guppy/ingot path. - No growth in stranded state: accepted-but-unregistered blobs converge to zero — registered, or reclaimed with a visible `Expired`/`Surplus`/`Removed` outcome — under fault injection: node crash before conclude, agent crash before conclude, sprue restart, `/blob/remove` racing a late delivery in both directions, and delivery after the horizon. +- Exactly one `/blob/register` receipt exists per registered acceptance under concurrent node and agent delivery. - The legacy single-blob flow is byte-identical on the wire before and after deployment. ## References From e05f2723f5cae7267f4f004544dcdb90a4823bc9 Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Tue, 8 Sep 2026 20:45:05 -0700 Subject: [PATCH 5/7] rfc: multi-blob add: placement programs as pre-minted invocations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The batch-add response returns to the deployed container contract: per candidate, the promise-chained task set (allocate, put, accept, register), sprue-signed, no-nonce, window expiries — the agent relays rather than authors. /blob/reject stays the one key-bound delegation (policy pinning space+digest). Conclusion verification collapses to sprue's own signatures with the register task as batch marker and replica carrier. Adds /blob/batch/abort (cause termination + pending tombstones), the three blob-removal amendments stated in Compatibility, node-side refusal of register-less accepts, deferred PDP enqueue until registration signal, and the delegation model as Alternatives considered. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 262 ++++++++++++++++++++------------- 1 file changed, 161 insertions(+), 101 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index 6ca1ed9..bd37f6e 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -14,10 +14,10 @@ The clients that drive volume know every digest before the first network call. I This RFC proposes a batch upload protocol: -- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate nodes plus delegations that let the agent invoke `/blob/allocate` and `/blob/accept` on those nodes directly. -- The agent drives the entire node exchange itself: one container of `/blob/allocate` invocations per node, PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land, then one container of `/blob/accept` invocations per winning node. The accept receipts — location claim and PDP promise included — are in the agent's hands when the accept round completes. Nothing is polled and nothing is awaited from the upload service. -- Registration travels as receipt delivery over `/ucan/conclude`, which stays what it is today: an open conclusion mechanism that anyone holding a receipt may invoke. The storage node SHOULD conclude each accept it executes; the agent SHOULD deliver its collected receipts as well, batched. The upload service registers idempotently from whichever delivery arrives first. -- `/blob/batch/extend` — replacement candidates when a blob exhausts its set. +- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate placements, each carrying the upload's tasks as pre-minted, promise-chained invocations — `/blob/allocate`, `/http/put`, `/blob/accept`, `/blob/register` — exactly as today's `/blob/add` response carries them for one blob and one node. +- The agent drives the node exchange itself: it relays each candidate's allocate invocations in one container per node, PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land, then relays the accept invocations at the winners. The accept receipts — location claim and PDP promise included — are in the agent's hands when the accept round completes. Nothing is polled and nothing is awaited from the upload service. +- Registration travels as receipt delivery over `/ucan/conclude`, which stays what it is today: an open conclusion mechanism that anyone holding a receipt may invoke. The storage node SHOULD conclude each accept it executes; the agent SHOULD deliver its collected receipts as well, batched. The upload service executes the pre-minted `/blob/register` task idempotently from whichever delivery arrives first. +- `/blob/batch/extend` — replacement candidates when a blob exhausts its set; `/blob/batch/abort` — cancel a batch's outstanding placements. Cost for an M-blob object: one synchronous round trip to the upload service, one allocate and one accept round trip per node plus the PUTs, and asynchronous receipt deliveries that never sit on the completion path. @@ -25,13 +25,14 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one - One synchronous client↔upload-service exchange per batch, independent of blob count, plus one `/blob/batch/extend` exchange per repair round. - No polling and no conclusion wait: the agent holds every receipt it needs — accept, location claim, PDP promise — directly from the nodes, at the end of the node exchange. +- The response container carries the upload as executable tasks: each candidate placement is a complete, sprue-signed program, every argument fixed, nothing for a holder to vary. The one right outside the program, early unwind via `/blob/reject`, travels as a key-bound delegation. - `/ucan/conclude` remains an intentionally open delivery mechanism; the upload service trusts what the delivered chain proves and nothing about who delivered it. -- Reporting duty sits with the party that bears the loss. An accepted blob nobody registers is unpaid bytes on the node, so the node concludes its own accepts; the agent's delivery is the second, independent path. -- Every acceptance has a protocol-visible registration horizon, derived from the delegation that authorized it, so acceptance, registration and reclaim can never disagree about whether a blob is live. -- The upload service drops off the upload path entirely after planning. It mints capabilities without contacting a node and never fans out to nodes at all. -- Every node-side action remains traceable to a client-signed intent: the allocate's `cause` names the client's batch invocation, and the delegation policy binds it. +- Reporting duty sits with the party that bears the loss: the node concludes its own accepts, and the agent's delivery is the second, independent path. +- Every acceptance has a protocol-visible registration horizon, read off the expiry of the delivered task itself, so acceptance, registration and reclaim can never disagree about whether a blob is live. +- The upload service drops off the upload path entirely after planning. It mints the tasks without contacting a node and never fans out to nodes at all. +- Every node-side action is linked to the client's intent: the allocate's `cause` names the client's batch invocation. - Partial success is first-class. Blobs succeed and fail independently; delivery is idempotent and repeatable, so stragglers converge under the same batch. -- The receipt chain keeps today's shape — `add → allocate → put → accept` — so everything that walks receipts (replay, removal routing) is unchanged, and gains a sprue-signed terminal link: registration itself becomes a chain-visible `/blob/register` receipt. +- The receipt chain keeps today's shape — `add → allocate → put → accept` — and gains a sprue-signed terminal link: registration itself becomes a chain-visible `/blob/register` receipt. ## Concepts @@ -45,52 +46,37 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one | Space | A namespace (cryptographic key pair) that owns content. A bucket is a space. | | Ingot | An S3 facade typically co-located with a Forge Piri node. | -### The third delegation regime: re-delegated placement +### The placement program -The existing upload protocol has two delegation regimes: space-rooted (the space delegates upload capabilities to the agent, which invokes on sprue) and provider-rooted (the node delegates `/blob/allocate` and `/blob/accept` to sprue at registration, and sprue invokes on the node). This RFC adds a third, composed from the second: sprue re-delegates `/blob/allocate` and `/blob/accept` to the agent — issuer sprue, audience the agent, subject the **provider DID** — with a policy that pins the invocation arguments to exactly the placement sprue decided: +Today's `/blob/add` response already carries the upload as a program: the `/blob/allocate`, `/http/put` and `/blob/accept` invocations, promise-chained, each executed by a different party. This RFC keeps that contract and widens it. For each candidate placement of each blob, sprue pre-mints the full task chain: -```jsonc -// Policy on each placement delegation (dag-json for readability) -[ - ["==", ".space", "did:key:zSpace"], - ["==", ".blob.digest", { "/": { "bytes": "EiCcvAfD..." } }], - ["==", ".blob.size", 1048576], - ["==", ".cause", { "/": "bafy...batchaddtask" }] -] -``` - -The agent invokes the node verbs under the chain provider → sprue → agent. Validation needs nothing new: ucantone's validator checks the subject and applies delegation policies conjunctively at every hop, so the agent can act only on the announced digest, at the announced size, for the named space and cause, at the nodes sprue selected, until the delegation expires. Escalation is rejected by the existing validator; there is no issuer allow-list on these handlers to bypass. +| Task | Executor | Chaining | +| --- | --- | --- | +| `/blob/allocate` | the node (the agent relays the invocation) | `cause` = the `/blob/batch/add` task | +| `/http/put` | the client (signs the receipt with the digest-derived key sprue embeds, as today) | `destination` awaits the allocate task | +| `/blob/accept` | the node (the agent relays the invocation) | `_put` awaits the put task | +| `/blob/register` | sprue, at delivery | `accept` links the accept task | -Two facts of each placement delegation do double duty later. Its metadata carries the blob's **replica count** (one, unless placement says otherwise), which bounds registration. Its **expiry** defines the registration horizon: the chain it roots can register only until that expiry plus a protocol grace period (see [Registration by receipt delivery](#registration-by-receipt-delivery)). +The allocate and accept invocations are issued and signed by sprue with the provider as subject, under the provider's registration delegation — the same chain nodes validate today; sprue likewise authors the put task under the digest-derived key, and the sprue-subject register task. The agent adds no authority and can vary nothing, only execute or not execute. All four are built without nonces, so their task CIDs are deterministic and known to every party from the moment the response lands, including the register task, whose receipt the client can await from t=0. The allocate, put and accept tasks carry an explicit expiry equal to the batch's upload window (the 30-second invocation default would be useless here). The register task expires later, at the registration horizon: the window plus a delivery allowance for receipts to land. -Handing over allocate is safe: it is idempotent, keyed `(digest, space)`, and an allocation that never receives bytes is parked state the node's existing expiry reclaims. Handing over accept is safe because registration follows the receipt rather than the invoker: acceptance creates the durable obligations — a published location claim, a PDP enqueue — and what keeps those from dangling is not who invoked accept but that the accept receipt reaches the upload service inside the horizon, which the node itself has the strongest incentive to ensure. +Because allocate's subject is the provider, the chain is minted per candidate: a blob with three candidates gets three parallel programs, of which the client executes one (or more, up to the replica count). Unexecuted programs cost nothing and expire. -The baseline is one delegation pair per blob per candidate node. For large batches sprue MAY compact to one pair per provider, binding the digest set with the policy language's `or` operator. Sprue MAY additionally include a policy-bound `/blob/reject` delegation per placement, letting the agent retire its own parked pieces early instead of waiting for allocation expiry; this is safe because reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). +One artifact in the placement is not an invocation: a **`/blob/reject` delegation**, audience the agent, its policy pinning the space and the digest so the agent can unwind only its own placement's parked state — content-addressed dedup means another tenant can hold a parked allocation for the same digest, and an unpinned space field would reach it. Reject unwinds the upload rather than advancing it, and that difference is the security line: the program's tasks are idempotent and produce only what the space announced, so they are safe to execute on delivery no matter who carries them, while reject carries destructive discretion over in-flight state and is therefore key-bound to the agent. It is safe against anything accepted: reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). ### Over-allocation and the race Placement returns more candidates than a blob needs. The agent allocates at its candidates, PUTs to as many as it likes, and stops when enough land — the race is client policy, from "primary first, spare on failure" to "start all, cancel at first." A candidate whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and is an instant winner. -The agent then invokes `/blob/accept` at winners, and MUST accept at no more than the blob's replica count of them: acceptance is the durable state, and registration will not pay for more. A retried accept that lands late anyway can still leave one extra acceptance; that converges rather than corrupts — the surplus node learns its status from its own delivery outcome and reclaims, readers who hit its stale location claim retry the registered node's claim, and the agent reconciles its index from the delivery outcomes. +The agent then relays `/blob/accept` at winners, and MUST accept at no more than the blob's replica count of them: acceptance is the durable state, and registration will not pay for more. A retried accept that lands late anyway can still leave one extra acceptance; that converges rather than corrupts — the surplus node learns its status from its own delivery outcome and reclaims, readers who hit its stale location claim retry the registered node's claim, and the agent reconciles its index from the delivery outcomes. When relaying an accept, the agent MUST include the candidate's register invocation, put invocation and put receipt in the same container: those artifacts are what the node's own delivery needs, and the node refuses the accept without them. Losing candidates need no message at all: their parked allocations, and any bytes from PUTs cancelled after completion, expire node-side. -### The client-minted `/http/put` - -Today sprue builds the `/http/put` invocation during `/blob/add`, deriving an ed25519 key from the blob digest and embedding it in the invocation metadata so the client can sign the receipt. In this protocol the agent drives the node exchange, so the agent builds the put invocation itself — same derivation, same shape: - -- key: the multihash's 32-byte digest as an ed25519 seed; issuer = audience = subject = the derived `did:key`; -- arguments: the blob and a `destination` await linking the agent's own `/blob/allocate` task; -- receipt: self-signed with the derived key on PUT success — immediately, with no PUT, when the allocate receipt carried no upload address. - -Nothing about the artifact's strength changes — anyone holding the digest could always mint it, which is the deliberate design — only where it is minted. The agent's `/blob/accept` invocation links the put task in `_put`, so the chain `batch add → allocate → put → accept` is exactly today's chain shape. - ### Batch lifecycle -1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the placement delegations, and returns placements. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. -2. **Allocate, race, accept** — one container of `/blob/allocate` invocations per candidate node; PUTs in parallel; cancel at enough; one container of `/blob/accept` invocations per winning node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. -3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, mints the `/blob/register` receipt idempotently, and answers with per-receipt outcomes linking it. -4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and its receipts are delivered later under the same batch. +1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the placement programs, and returns them. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. +2. **Allocate, race, accept** — one container of allocate invocations per candidate node; PUTs in parallel; cancel at enough; one container of accept invocations per winning node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. +3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, executes the pre-minted `/blob/register` task idempotently, and answers with per-receipt outcomes linking its receipt. +4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and its receipts are delivered later under the same batch. A batch the client walks away from is cancelled with `/blob/batch/abort`. ### Transport @@ -104,7 +90,7 @@ Batching travels as ucantone container contents: a single HTTP exchange carries * Audience: Sprue * Subject: The space -Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob with the minted delegations carried in the response container. Candidates come only from nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob, with the placement programs' invocations and the reject delegations carried in the response container alongside the provider registration delegations the node-facing invocations need as proofs. Candidates come only from nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. @@ -161,7 +147,11 @@ e.g. type Placement struct { digest Bytes # multihash of the blob provider String # DID of the candidate storage node - proofs [Link] # delegations in the response container: /blob/allocate, /blob/accept, optionally /blob/reject + allocate Link # the /blob/allocate invocation, in the response container + put Link # the /http/put invocation + accept Link # the /blob/accept invocation + register Link # the /blob/register invocation, executed by sprue at delivery + reject Link # the /blob/reject delegation, for early unwind } type BatchAddOK struct { @@ -178,7 +168,11 @@ type BatchAddOK struct { type Placement struct { Digest multihash.Multihash `cborgen:"digest"` Provider did.DID `cborgen:"provider"` - Proofs []cid.Cid `cborgen:"proofs"` + Allocate cid.Cid `cborgen:"allocate"` + Put cid.Cid `cborgen:"put"` + Accept cid.Cid `cborgen:"accept"` + Register cid.Cid `cborgen:"register"` + Reject cid.Cid `cborgen:"reject"` } type BatchAddOK struct { @@ -197,20 +191,20 @@ type BatchAddOK struct { The existing conclusion command, extended to carry many receipts. Conclude is deliberately an **open receipt-delivery mechanism**: it carries no authority of its own, anyone may invoke it, and the upload service acts only on what the delivered artifacts prove. Piri already uses it outbound this way, delivering its replica-transfer receipts to the upload service. The arguments gain an optional plural field — exactly one of `receipt` or `receipts` MUST be present — and a server implementing this RFC MUST accept both forms; the legacy single-receipt flow is untouched. -For a delivered `/blob/accept` receipt to register a blob, the request container MUST make the full chain resolvable: the accept invocation, its node-signed receipt, the put invocation its `_put` field awaits, and the `/blob/allocate` invocation that put's `destination` awaits. Deliverers SHOULD include the allocate receipt as well, so sprue persists a complete chain for receipt-chain consumers. +For a delivered `/blob/accept` receipt to register a blob, the request container MUST make the placement program resolvable: the accept invocation, its node-signed receipt, the put invocation its `_put` field awaits, the `/blob/allocate` invocation that put's `destination` awaits, and the placement's `/blob/register` invocation. Deliverers SHOULD include the allocate receipt as well, so sprue persists a complete chain for receipt-chain consumers. Sprue dispatches each delivered receipt whose `Ran` command has a registered conclusion handler; other artifacts in the container are context. The new `/blob/accept` conclusion handler, per delivery: -1. verifies the accept receipt is signed by the provider DID that is the accept invocation's subject, with `Ran` matching and a success result; -2. verifies the accept invocation's proof chain terminates in a **placement delegation sprue itself signed**, whose subject is the provider and whose policy pins the space, digest, size and cause. Sprue verifying its own signature is the placement check, and it requires no stored state; -3. enforces the registration horizon: a chain whose placement delegation expired more than the protocol grace period ago MUST be refused with an `Expired` outcome. Within the horizon, freshness is not otherwise rechecked — the node-signed receipt attests that the chain, expiry included, validated at execution time; -4. enforces the registration bound: at most the replica count the placement delegation's metadata names — one, absent the field — is registered per `(digest, space)`. The earliest delivered acceptances win, up to the bound; further acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; +1. verifies the accept receipt is signed by the provider DID that is the accept invocation's subject, with `Ran` matching and a success result — else `InvalidReceipt`; +2. verifies the accept and register invocations are **issued and signed by sprue itself**, and that the register invocation's `accept` field names the delivered accept task. Sprue verifying its own signatures is the placement check, and it requires no stored state — the register task exists only for placements sprue minted, which also keeps legacy sprue-signed accepts out of this path — failures return `UnknownPlacement`; +3. enforces the registration horizon: a register task that expired more than the protocol grace period ago MUST be refused with an `Expired` outcome. Within the horizon, freshness is not otherwise rechecked — the node-signed receipt attests that the accept validated, expiry included, at execution time; +4. enforces the registration bound: at most the replica count the register invocation names is registered per `(digest, space)`. The earliest delivered acceptances win, up to the bound; further acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; 5. checks the tombstones (below): a chain matching one returns a `Removed` outcome instead of registering; -6. registers the blob by minting one [`/blob/register`](#blobregister) task per winning acceptance and issuing its receipt, debits the space's quota, and persists the delivered chain and the registration alongside it. +6. registers the blob by executing the register task — issuing its receipt — debits the space's quota, and persists the delivered chain and the registration alongside it. -Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` tasks and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement delegation was issued before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). Both checks read only the delivered delegation, so verification stays stateless. +Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` tasks and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement was minted before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). `/blob/batch/abort` writes the same pending tombstones for a whole batch at once. Both checks compare the delivered artifacts — the register task's `cause` and issuance time — against the tombstone record alone, so sprue keeps no journal of in-flight uploads. -Delivery is idempotent and unordered: the `/blob/register` task is built with no nonce, so the node's conclusion and the agent's conclusion of the same accept converge on the identical task CID and one receipt, and a repeat delivery returns the same registration link. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else, and a `Surplus` outcome still carries the blob's registration links, so the deliverer learns the registered providers directly. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. +Delivery is idempotent and unordered: the register task's CID is fixed at minting, so the node's conclusion and the agent's conclusion of the same accept converge on the identical task and one receipt, and a repeat delivery returns the same registration link. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else, and a `Surplus` outcome still carries the blob's registration links, so the deliverer learns the registered providers directly. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. Implementation note: the container accessor for receipts is a linear scan. Matching M deliveries to their container artifacts naively is O(M²); index the container's contents by link and by `Ran()` once before the loop. @@ -239,9 +233,8 @@ type ConcludeArguments struct { e.g. ```jsonc -// Encoded as dag-json for readability; the referenced accept receipts, -// accept invocations, put invocations and allocate invocations travel -// in the same container +// Encoded as dag-json for readability; the referenced accept receipts and +// their placement-program invocations travel in the same container { "iss": "did:key:zPiriNode", "aud": "did:web:up.forge.example.com", @@ -297,11 +290,11 @@ type BatchConcludeOK struct { * Audience: Sprue * Subject: Sprue -The terminal link of the upload chain, and a virtual task in the tradition of `/http/put`: nothing serves it. Sprue mints it inside the `/blob/accept` conclusion handler when a delivery verifies, issues the receipt itself, and persists both with the delivered chain. The receipt is the canonical, sprue-signed record that a blob is registered — the artifact billing, index reconciliation and disputes start from — and the registration time is the receipt's issue time. +The terminal task of the placement program, and a virtual task in the tradition of `/http/put`: no node serves it. Sprue mints it at `/blob/batch/add` alongside the other three tasks and executes it — issues its receipt — inside the `/blob/accept` conclusion handler when a delivery verifies. The receipt is the canonical, sprue-signed record that a blob is registered — the artifact billing, index reconciliation and disputes start from — and the registration time is the receipt's issue time. Sprue sets `replicas` from the space's placement policy at planning time; extend-minted register tasks carry the original batch's bound. Because the task CID is fixed at minting, the client can await this receipt from the moment the batch-add response lands. -The task is built with no nonce, so its CID is deterministic over its arguments: every delivery of the same acceptance names the identical task, and exactly one receipt exists per registered acceptance no matter who delivered first or how many times. Removing a registered blob retires its register task per acceptance; the cause-keyed tombstone records them. +Exactly one receipt exists per registered acceptance, no matter who delivered first or how many times. Removing a registered blob retires its register tasks; the cause-keyed tombstone records them. -Idempotent: a delivery that finds the registration already minted returns the existing receipt's link in its outcome. +Idempotent: a delivery that finds the registration already executed returns the existing receipt's link in its outcome. #### Arguments @@ -313,7 +306,8 @@ type RegisterArguments struct { blob Blob # digest and size provider String # DID of the storage node holding the registered acceptance cause Link # the /blob/batch/add task - accept Link # the /blob/accept task whose delivery triggered registration + accept Link # the /blob/accept task whose delivery triggers registration + replicas Int # registration bound for this blob, per (digest, space) } ``` @@ -327,6 +321,7 @@ type RegisterArguments struct { Provider did.DID `cborgen:"provider"` Cause cid.Cid `cborgen:"cause"` Accept cid.Cid `cborgen:"accept"` + Replicas uint64 `cborgen:"replicas"` } ``` @@ -341,9 +336,9 @@ Successful registration returns a unit result (`{}`). The receipt's substance is * Audience: Sprue * Subject: The space -Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK` — fresh candidates with fresh delegations, opening fresh quota holds and a fresh registration horizon for the re-placed blobs — and the returned delegations bind `cause` to the **original** batch-add task, so the whole batch delivers under one cause. +Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK`: fresh placement programs with fresh expiries and a fresh registration horizon for the re-placed blobs, whose quota holds replace the superseded candidates' outstanding ones, and whose tasks bind `cause` to the **original** batch-add task, so the whole batch delivers under one cause. Extend MUST NOT issue a placement for a digest carrying a live cause-keyed tombstone under this cause, and extending an aborted batch MUST fail with the named error `BatchAborted`. -Idempotent: extending a digest that has since been accepted or registered simply returns no placement for it, listed in `registered`. +Idempotent: extending a digest that has since registered simply returns no placement for it, listed in `registered`. #### Arguments @@ -402,17 +397,74 @@ e.g. `BatchAddOK`, as for [`/blob/batch/add`](#blobbatchadd). -### The node legs: `/blob/allocate` and `/blob/accept`, unchanged verbs +### `/blob/batch/abort` * Issuer: Agent +* Audience: Sprue +* Subject: The space + +Cancels a batch's outstanding placements: the client renounces every placement of the batch that has not registered. Sprue releases the batch's remaining quota holds and writes the pending tombstones immediately, so chains from the abandoned placements return `Removed` instead of registering — the abort is, in effect, revocation of the outstanding placement programs. Blobs already registered are untouched and listed in the result; they exit via `/blob/remove`. + +Abort is the upload-service half of cancellation. For node-side cleanup the client invokes `/blob/reject` under the placements' reject delegations, one container per node it touched, and SHOULD abort first so the tombstones are in place before any in-flight delivery races the cleanup. Acceptances that already executed cannot be rejected (`BlobAccepted`); they resolve through the delivery loop — the node concludes, receives `Removed`, and reclaims. + +Abort terminates the cause: a later `/blob/batch/extend` under it fails with `BatchAborted`, so a cancelled batch cannot mint post-tombstone placements that slip past its own cancellation. + +Idempotent: aborting an unknown, already-aborted, or fully-registered batch MUST succeed. + +#### Arguments + +**IPLD schema** + +```ipldsch +type BatchAbortArguments struct { + cause Link # the /blob/batch/add task being cancelled +} +``` + +
+Go syntax + +```go +type BatchAbortArguments struct { + Cause cid.Cid `cborgen:"cause"` +} +``` +
+ +#### Result + +**IPLD schema** + +```ipldsch +type BatchAbortOK struct { + cancelled [Bytes] # digests whose placements are tombstoned + registered [Bytes] # digests already registered; use /blob/remove +} +``` + +
+Go syntax + +```go +type BatchAbortOK struct { + Cancelled []multihash.Multihash `cborgen:"cancelled"` + Registered []multihash.Multihash `cborgen:"registered"` +} +``` +
+ +### The node legs: `/blob/allocate` and `/blob/accept`, unchanged verbs + +* Issuer: Sprue * Audience: Piri * Subject: The provider -The storage-node verbs, their argument types and their results are exactly today's. What changes: +The storage-node verbs, their argument types, their results, and their issuer are exactly today's — the invocations are sprue-signed under the provider's registration delegation, and only the courier changes: the agent relays them, with the proofs, in its own containers. `cause` on allocate names the `/blob/batch/add` task, so every allocation and acceptance a node writes names the client invocation that motivated it, and `_put` on accept awaits the placement's `/http/put` task, preserving the chain shape. -- Both are invoked by the agent under the placement delegations, with `cause` = the `/blob/batch/add` task CID (bound by policy), so every allocation and acceptance a node writes names the client invocation that motivated it. -- `/blob/accept`'s `_put` field awaits the client-minted `/http/put` task, preserving the chain shape. -- A node that serves batch placements advertises **batch support** at registration, which commits it to two behaviors. It SHOULD deliver each accept it executes to the upload service via `/ucan/conclude` (see [Registration by receipt delivery](#registration-by-receipt-delivery)), retrying until acknowledged. And it reclaims acceptances that remain unregistered past its reclaim deadline — learned from its delivery outcomes (`Surplus`, `Removed`, `Expired`) or from sustained delivery failure — where the deadline MUST be at least the placement delegation's expiry plus the protocol grace period, so reclaim can never race a chain that could still register. Beyond that floor the deadline is node policy. +A node that serves batch placements advertises **batch support** at registration, which commits it to two behaviors: + +- It MUST refuse to execute an accept whose container lacks the placement's register invocation, put invocation and put receipt — without them its acceptance could never deliver a registrable chain — and SHOULD deliver each accept it executes to the upload service via `/ucan/conclude` (see [Registration by receipt delivery](#registration-by-receipt-delivery)), retrying until acknowledged. It SHOULD defer enqueueing the acceptance into PDP aggregation until a registration signal — its delivery outcome, or the register receipt — so a forced or surplus acceptance costs storage until reclaim rather than on-chain proof obligations. +- It reclaims acceptances that remain unregistered past its reclaim deadline — learned from its delivery outcomes (`Surplus`, `Removed`, `Expired`, `UnknownPlacement`) or from sustained delivery failure — where the deadline MUST be at least the register task's expiry plus the protocol grace period, so reclaim can never race a chain that could still register. Beyond that floor the deadline is node policy. Reclaim has the semantics of a self-invoked `/blob/release` under the [blob-removal RFC](./2026-07-forge-blob-removal.md): it drops that space's acceptance and location claim, and physical deletion remains gated on zero claims across spaces and on-chain proof retirement. This is a deliberate amendment to that RFC's lifecycle, which otherwise lets an acceptance exit only via `/blob/remove`; the exit exists only for acceptances that never registered. ## Registration by receipt delivery @@ -420,81 +472,89 @@ Acceptance happens at the node; registration happens at sprue; and the bridge be - **The node holds the receipt it minted and bears the loss if registration never happens** — an accepted blob nobody registers is bytes it stores and proves unpaid. So the node SHOULD conclude every accept it executes, retrying until acknowledged. - **The agent holds the same receipts** and SHOULD deliver them batched — typically immediately, as its confirmation that registration is underway. Either delivery alone suffices; both together make loss require two independent failures. -- **Sprue does only local work per delivery**: verify the chain, apply the horizon, the bound and the tombstones, mint the registration, persist. Registration is idempotent, so redundant and repeated deliveries converge on one `/blob/register` receipt, and a delivery can be replayed at any time by anyone still holding the receipts. +- **Sprue does only local work per delivery**: verify its own signatures on the program, apply the horizon, the bound and the tombstones, execute the register task, persist. Registration is idempotent, so redundant and repeated deliveries converge on one `/blob/register` receipt, and a delivery can be replayed at any time by anyone still holding the receipts. -The **registration horizon** is what keeps the three parties' clocks consistent. A chain registers only until its placement delegation's expiry plus a protocol grace period; the node's reclaim deadline starts at that same boundary. So a chain that can still register names bytes the node still holds, a reclaimed acceptance names a chain that can no longer register, and a months-old receipt delivered by anyone is refused rather than resurrected as a billed ghost. The failure this makes visible instead of silent: a delivery that misses the horizon returns `Expired`, the node reclaims, and the client re-uploads under a fresh batch — rather than sprue registering bytes no node holds. +The **registration horizon** is what keeps the three parties' clocks consistent, and it is read off the delivered artifact itself: a chain registers only until its register task's expiry plus a protocol grace period, and the node's reclaim deadline starts at that same boundary. So a chain that can still register names bytes the node still holds, a reclaimed acceptance names a chain that can no longer register, and a months-old receipt delivered by anyone is refused rather than resurrected as a billed ghost. A delivery that misses the horizon returns `Expired`, the node reclaims, and the client re-uploads under a fresh batch; sprue never registers bytes no node holds. The residual window is bounded: a blob is accepted-but-unregistered until the first delivery lands, and if every holder crashes before delivering, until one recovers or the horizon closes it out. -The client's completion does not wait for any of this. Resolvability comes from accept itself (the node publishes its location claim synchronously, as today), so the S3 200 returns on accept receipts; registration — billing, retain seeding, repair and removal routing — converges with delivery, typically within the agent's own immediate conclude. A client that wants a stronger signal than the 200 holds one artifact: the `/blob/register` receipt linked from its delivery outcome, verifiable offline and presentable to anyone. +The client's completion does not wait for any of this. Resolvability comes from accept itself (the node publishes its location claim synchronously, as today), so the S3 200 returns on accept receipts; registration — billing, retain seeding, repair and removal routing — converges with delivery, typically within the agent's own immediate conclude. A client that wants a stronger signal than the 200 holds one artifact: the `/blob/register` receipt, addressable by task CID from the batch-add response, verifiable offline and presentable to anyone. ## Failure handling and idempotency - A failed or stalled PUT races another candidate; no sprue call. An exhausted candidate set costs one `/blob/batch/extend`. - A refused accept is visible to the agent directly in the node's response; the blob goes back through the race or through extend. -- An `Expired` outcome means the batch's registration window lapsed undelivered; the blob re-uploads under a fresh batch. -- The agent SHOULD reconcile its index against the `/blob/register` receipt linked from its delivery outcomes: it names the registered provider, whose location claim is the durable one. -- Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. -- Every capability in this RFC is idempotent: re-adding, re-delivering and re-extending converge on the same state, so the retry story for the whole protocol is "re-invoke". +- An `Expired` outcome means the batch's registration horizon lapsed undelivered; the blob re-uploads under a fresh batch. +- The agent SHOULD reconcile its index against the `/blob/register` receipts linked from its delivery outcomes: they name the registered providers, whose location claims are the durable ones. +- Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. An abandoned batch is cancelled sooner with `/blob/batch/abort` plus reject. +- Every capability in this RFC is idempotent: re-adding, re-delivering, re-extending and re-aborting converge on the same state, so the retry story for the whole protocol is "re-invoke". + +## Example flow + +An object of M blobs raced across N candidate nodes: + +| # | Leg | What moves | +|---|-----|------------| +| 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | +| 2 | sprue → agent | `BatchAddOK`: candidate placement programs in the container | +| 3 | agent → each node, parallel | one container relaying that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — warm, instant winner) | +| 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; sign a put receipt per winner | +| 5 | agent → each winning node, parallel | one container relaying that node's `/blob/accept` invocations (register and put invocations and put receipts included); receipts carry location claims and PDP promises — **the client is done** | +| 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | +| 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response links a sprue-signed `/blob/register` receipt per registered acceptance | + +A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs that failed everywhere, repeats legs 3–5 against the fresh candidates, and the late receipts ride the same delivery path under the original cause. A batch the client abandons is cancelled with `/blob/batch/abort` followed by `/blob/reject` at the touched nodes. + +Synchronous round trips: one to sprue per batch, two per node — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 41 synchronous control exchanges, all parallel across nodes, plus at most 21 asynchronous receipt deliveries off the completion path — instead of ~500, serial per blob. Byte PUTs are excluded from both counts. ## Security considerations -- **Attenuation.** The placement delegations' policy pins space, digest, size and cause; ucantone applies policies conjunctively at every hop and rejects escalation. An agent holding a batch's delegations can act only on the placements sprue decided to permit, nowhere else, and only until expiry. -- **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt whose invocation chain terminates in sprue's own placement delegation, inside the horizon. The conclude issuer is irrelevant by design — a leaked receipt chain delivered by a stranger causes at most a correct, in-horizon registration of the space the chain names. Registration produces evidence in turn: the sprue-signed `/blob/register` receipt is the record billing and disputes start from. -- **No new agent trust.** The agent gains no authority it could not already exercise through `/blob/add`; sprue's placement discretion is unchanged, exercised at plan time instead of invocation time. -- **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that over-accepts gains nothing durable and costs the nodes only until reclaim. +- **The program is fixed.** Every node-facing artifact is a complete sprue-signed invocation: all arguments set, nothing for the holder to vary. The agent's entire latitude is which pre-authorized programs to execute. +- **Bearer execution is bounded by the announcement.** The invocations execute on delivery, so a holder of a leaked response container can advance the program — at worst force-completing an announced upload of content it possesses, billing the space for blobs and sizes the space itself named, inside the expiry window. It cannot divert the program to other digests, sizes, spaces, or non-candidate nodes; within a blob's candidate set a faster delivery can pick a different winner than the agent chose, which converges through `Surplus` and reclaim like any late acceptance. `/blob/batch/abort`'s tombstones close even the forced-completion sliver for a batch the client cancels. The one destructive right, reject, is key-bound to the agent precisely because a bearer form of it would let an interceptor destroy in-flight uploads. +- **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt for an accept invocation sprue itself signed, named by a register task sprue itself signed, inside the horizon. The conclude issuer is irrelevant by design. Registration produces evidence in turn: the sprue-signed `/blob/register` receipt is the record billing and disputes start from. +- **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that over-accepts gains nothing durable and costs the nodes storage until reclaim — and, on nodes that defer PDP enqueue to the registration signal, no proof obligations. - **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and nothing rests on it; the node-signed accept receipt is the artifact registration relies on. -- **Replay.** Delegations expire; invocations carry nonces; allocation and acceptance are keyed `(digest, space)`; delivery, registration and reclaim all converge under repeats; the horizon refuses stale chains outright, and the tombstones stop a replayed delivery from resurrecting a removed registration in either race direction. -- **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint delegations for M blobs, and one conclude makes it verify M chains. Sprue SHOULD enforce configurable caps on blobs per batch and receipts per conclude. Quota SHOULD be taken as a hold with a TTL equal to the delegation expiry, released on expiry and reconciled at registration; registration inside the horizon debits unconditionally, so the over-commit exposure is bounded to chains delivered in the grace period after their hold released — an outright debit at add would leak on abandoned batches, and a bare check would over-commit across parallel batches. +- **Replay.** Invocation task CIDs are deterministic and their effects idempotent, keyed `(digest, space)`; delivery, registration and reclaim all converge under repeats; the horizon refuses stale chains outright, and the tombstones stop a replayed delivery from resurrecting a removed registration in either race direction. +- **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint programs for M blobs, and one conclude makes it verify M chains. Sprue SHOULD enforce configurable caps on blobs per batch and receipts per conclude. Quota SHOULD be taken as a hold with a TTL equal to the registration horizon, released on expiry or abort and reconciled at registration; registration inside the horizon debits unconditionally, so the over-commit exposure is bounded to chains delivered in the grace period after their hold released — an outright debit at add would leak on abandoned batches, and a bare check would over-commit across parallel batches. ## Limits -The cbor-gen list cap (8,192 elements) bounds every list in these schemas. The per-candidate delegation baseline puts two or three signed delegations per candidate, a few hundred bytes each, in the batch-add response container — order of a megabyte per thousand candidates. Batches large enough to care SHOULD use the per-provider `or`-compacted delegation form. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. A placement is five signed artifacts — four invocations and a delegation — a few hundred bytes each, so the batch-add response runs to a couple of megabytes per thousand candidates; batch and candidate caps size accordingly. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. ## Compatibility -- **Storage nodes**: the verbs, argument types and results are unchanged, and the provider → sprue → agent chain validates under existing subject-plus-policy checking. Outbound conclusion has an existing precedent — piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. -- **`/blob/add`**: remains, unchanged, for single-blob callers and for streaming clients that cannot know digests up front. A streaming client MAY still batch per flush window — announce the blobs whose digests it has, race, accept, deliver, repeat. +- **Storage nodes**: the verbs, argument types, results and issuer are unchanged — nodes validate today's exact chain; only the courier differs. Outbound conclusion has an existing precedent: piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. +- **`/blob/add` and the [blob protocol spec](https://github.com/fil-forge/ucan-protocol-specs/blob/main/blob.md)**: the single-blob flow remains, unchanged, for callers that cannot know digests up front, and the spec's response-container contract generalizes rather than changes — the container still carries the upload's promise-chained task invocations; there are simply more of them, plus the explicit terminal register task. A streaming client MAY batch per flush window — announce the blobs whose digests it has, race, accept, deliver, repeat. - **`/ucan/conclude`**: the plural field is additive, and the `/blob/accept` conclusion handler is a new entry in the existing dispatch map; existing single-receipt clients and the legacy `/http/put` handler are untouched. -- **Receipt-chain consumers**: the persisted chain — `batch add → allocate → put → accept`, closed by a sprue-signed `register` — has today's shape in its prefix, so the replay path and the [blob-removal RFC](./2026-07-forge-blob-removal.md)'s receipt-chain routing work unchanged for delivered blobs; consumers MAY also start from the registration receipt, which links the accept task directly. Undelivered blobs differ: sprue has not seen their chains, so `/blob/abort` cannot route for them. The backstop is node-side allocation expiry and the reclaim deadline; the optional `/blob/reject` delegation gives an agent early reclaim directly against the node. +- **The [blob-removal RFC](./2026-07-forge-blob-removal.md)** is amended in three stated places. Its receipt-chain routing walk (the registration's cause, then the add receipt's `site`, then the accept task) does not exist for batch registrations — `BatchAddOK` has no `site` — so for batch-registered blobs routing MUST start from the `/blob/register` receipts, which name the providers and link the accept tasks directly, and `/blob/remove` fans `/blob/release` to every registered acceptance's provider. Its remove handler gains duties: the pending tombstone on removal of an unregistered digest, and per-`(digest, space)` serialization with registration. And its lifecycle gains the reclaim exit for never-registered acceptances, as specified in [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs). `/blob/abort` cannot route for batch blobs (sprue never learns which candidates the client executed until delivery); `/blob/batch/abort` plus the reject delegations replace it at batch granularity, with node-side expiry as the passive backstop. - **Read-on-write**: resolvability is preserved at its source — the node publishes its location claim synchronously inside accept, before the agent ever holds the receipt. Registration-dependent behavior converges with delivery. Index and upload registration (`/index/add`, `/upload/add`) follow the accept receipts exactly as today, and are out of scope here. -## Future work +## Alternatives considered -- **Fold the node legs into the PUT.** The end state carries the write authorization in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it — the delegation regime and the delivery model are the same. +### Policy-bound delegations instead of pre-minted invocations -## Example flow +The batch-add response could carry authority instead of tasks: sprue re-delegates `/blob/allocate` and `/blob/accept` to the agent — subject the provider, policy pinning space, digest, size and cause — and the agent authors the node-facing invocations itself. The two shapes are close: the artifact count per candidate is similar, the delivery, horizon, tombstone and advertisement machinery is identical, and nodes validate a provider-rooted chain either way. -An object of M blobs raced across N candidate nodes: +What the delegation shape buys: the response container is key-bound, so an intercepted copy is inert without the agent's key, where pre-minted invocations execute on delivery (bounded to force-completing the announced upload, as the security section states); the agent's own signature appears on the node-facing chain, rooting every node action in the customer's intent chain rather than in sprue's authorship; and very large batches can compact to one delegation pair per provider with an `or` policy over the digest set, which invocations cannot imitate. -| # | Leg | What moves | -|---|-----|------------| -| 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | -| 2 | sprue → agent | `BatchAddOK`: candidates per blob, placement delegations in the container | -| 3 | agent → each node, parallel | one container batching that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — warm, instant winner) | -| 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; mint a put receipt per winner | -| 5 | agent → each winning node, parallel | one container batching that node's `/blob/accept` invocations; receipts carry location claims and PDP promises — **the client is done** | -| 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | -| 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response links a sprue-signed `/blob/register` receipt per registered blob | +What it costs, and why this RFC chose invocations: a policy pins only the fields it names, so policy completeness is a standing obligation — today's argument schemas already contain an unpinned field (`_put`), and every future argument added to a node verb silently becomes agent latitude until someone pins it — where an invocation is complete automatically; conclusion verification becomes a proof-chain walk with a policy-binding checklist instead of a self-signature check; and the response-container contract diverges from the deployed protocol's instead of generalizing it. A deployment that wants the agent-signed intent chain — a fleet whose constraint is that the validated chain be the customer's chain — takes this alternative wholesale; nothing else in the RFC changes. -A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs that failed everywhere, repeats legs 3–5 against the fresh candidates, and the late receipts ride the same delivery path under the original cause. +## Future work -Synchronous round trips: one to sprue per batch, two per node — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 41 synchronous control exchanges, all parallel across nodes, plus at most 21 asynchronous receipt deliveries off the completion path — instead of ~500, serial per blob. Byte PUTs are excluded from both counts. +- **Fold the node legs into the PUT.** The end state carries the placement program in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it — the program and the delivery model are the same. ## Open questions -- The value of the registration grace period, and of the delegation expiry it extends (the delegation lifetime is also the upload window, so it wants to be generous — hours, not seconds). +- The values of the upload window (the upload tasks' expiry), the delivery allowance that sets the registration horizon (the register task's expiry), and the grace period beyond it; the window wants to be generous, on the order of hours. - The mechanics of the batch-support advertisement at registration: a capability proven in the registration container, alongside the four commands required today, or a flag on the provider record. - Whether the agent awaits its conclude response — and with it the `/blob/register` receipt — before the S3 200, or fires it after; this RFC leans to the latter since resolvability does not depend on it. - Default and maximum sizes for the batch, conclude and candidate-set caps. -- Whether `/blob/reject` delegations ship in placement proofs by default or on request. -- Whether the per-provider `or`-compacted delegation form ships in v1 or waits for a batch size that needs it. ## Evaluation criteria - Synchronous control exchanges for an M-blob object drop from ~5M to 1 + 2N, measured on ingot's sharded-object upload, with zero polling. - End-to-end upload latency for a multi-blob object improves by at least the removed poll, conclusion wait and serialization overhead, measured against the current guppy/ingot path. -- No growth in stranded state: accepted-but-unregistered blobs converge to zero — registered, or reclaimed with a visible `Expired`/`Surplus`/`Removed` outcome — under fault injection: node crash before conclude, agent crash before conclude, sprue restart, `/blob/remove` racing a late delivery in both directions, and delivery after the horizon. +- No growth in stranded state: accepted-but-unregistered blobs converge to zero — registered, or reclaimed with a visible `Expired`/`Surplus`/`Removed` outcome — under fault injection: node crash before conclude, agent crash before conclude, sprue restart, `/blob/remove` racing a late delivery in both directions, abort racing an in-flight delivery, and delivery after the horizon. - Exactly one `/blob/register` receipt exists per registered acceptance under concurrent node and agent delivery. - The legacy single-blob flow is byte-identical on the wire before and after deployment. From f6e70ee39aa82791313f96b6a422178d65a5d5a1 Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Wed, 9 Sep 2026 02:18:12 -0700 Subject: [PATCH 6/7] rfc: multi-blob add: one placement per blob MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified against upstream Storj: over-provisioning is redundancy-space only (110 limits for 80 needed, one node per piece, no spares; replacement via an amendment RPC). The RFC drops the candidate-set/racing machinery accordingly: one node, one program per blob — the current protocol per blob, batched. Sharding clients over-provision in their own blob set and walk away from the tail; extend is the amendment analogue with cumulative client-supplied exclusions. Also removes the replicas field (bound is one; replication moved to Future work), collapses to a single expiry (window + grace = horizon), and scopes abort by digest so a tail can be renounced while its batch stays live. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 98 +++++++++++++++++----------------- 1 file changed, 48 insertions(+), 50 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index bd37f6e..516ff30 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -10,14 +10,14 @@ Status: Experimental Uploading one blob today costs the client two POSTs to the upload service, an HTTP PUT of the bytes to the storage node, and a receipt poll, and costs the upload service two invocations against the storage node, all serial: `/blob/add`, the PUT, `/ucan/conclude` carrying a self-issued `/http/put` receipt, then `GET /receipt/{cid}` until the `/blob/accept` receipt appears — at least one more round trip per blob, and a second of added latency whenever the receipt is not ready on the first try. An object of M blobs pays that chain M times. Client-side parallelism hides some of the latency but none of the message count. -The clients that drive volume know every digest before the first network call. Ingot spools a PutObject to disk and shards it before uploading; any client that pre-shards or erasure-codes an object holds its full digest list up front. Nothing in the protocol lets them say so. And the protocol has no vocabulary for racing an upload: one node is selected per blob, and if it stalls, the whole blob waits. +The clients that drive volume know every digest before the first network call. Ingot spools a PutObject to disk and shards it before uploading; any client that pre-shards or erasure-codes an object holds its full digest list up front. Nothing in the protocol lets them say so. And a sharding client cannot express the shape of its redundancy: it needs only enough of its pieces to land, but the protocol makes it drive every blob to completion, one at a time. This RFC proposes a batch upload protocol: -- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, a set of candidate placements, each carrying the upload's tasks as pre-minted, promise-chained invocations — `/blob/allocate`, `/http/put`, `/blob/accept`, `/blob/register` — exactly as today's `/blob/add` response carries them for one blob and one node. -- The agent drives the node exchange itself: it relays each candidate's allocate invocations in one container per node, PUTs in parallel to as many candidates as it chooses, cancelling the rest once enough land, then relays the accept invocations at the winners. The accept receipts — location claim and PDP promise included — are in the agent's hands when the accept round completes. Nothing is polled and nothing is awaited from the upload service. +- `/blob/batch/add` — announce M blobs in one invocation. The upload service plans placement for the whole batch in one pass and returns, per blob, one placement carrying the upload's tasks as pre-minted, promise-chained invocations — `/blob/allocate`, `/http/put`, `/blob/accept`, `/blob/register` — exactly as today's `/blob/add` response carries them for one blob and one node. +- The agent drives the node exchange itself: one container of allocate invocations per node, the PUTs in parallel, one container of accept invocations per node. The accept receipts — location claim and PDP promise included — are in the agent's hands when the accept round completes. Nothing is polled and nothing is awaited from the upload service. A sharding client over-provisions in its blob set — announce more pieces than the code needs, finish enough, walk away from the tail — which needs no protocol support beyond batching itself. - Registration travels as receipt delivery over `/ucan/conclude`, which stays what it is today: an open conclusion mechanism that anyone holding a receipt may invoke. The storage node SHOULD conclude each accept it executes; the agent SHOULD deliver its collected receipts as well, batched. The upload service executes the pre-minted `/blob/register` task idempotently from whichever delivery arrives first. -- `/blob/batch/extend` — replacement candidates when a blob exhausts its set; `/blob/batch/abort` — cancel a batch's outstanding placements. +- `/blob/batch/extend` — a replacement placement when a blob's node fails; `/blob/batch/abort` — cancel a batch's outstanding placements. Cost for an M-blob object: one synchronous round trip to the upload service, one allocate and one accept round trip per node plus the PUTs, and asynchronous receipt deliveries that never sit on the completion path. @@ -25,10 +25,9 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one - One synchronous client↔upload-service exchange per batch, independent of blob count, plus one `/blob/batch/extend` exchange per repair round. - No polling and no conclusion wait: the agent holds every receipt it needs — accept, location claim, PDP promise — directly from the nodes, at the end of the node exchange. -- The response container carries the upload as executable tasks: each candidate placement is a complete, sprue-signed program, every argument fixed, nothing for a holder to vary. The one right outside the program, early unwind via `/blob/reject`, travels as a key-bound delegation. +- The response container carries the upload as executable tasks: each blob's placement is a complete, sprue-signed program, every argument fixed, nothing for a holder to vary. The one right outside the program, early unwind via `/blob/reject`, travels as a key-bound delegation. - `/ucan/conclude` remains an intentionally open delivery mechanism; the upload service trusts what the delivered chain proves and nothing about who delivered it. - Reporting duty sits with the party that bears the loss: the node concludes its own accepts, and the agent's delivery is the second, independent path. -- Every acceptance has a protocol-visible registration horizon, read off the expiry of the delivered task itself, so acceptance, registration and reclaim can never disagree about whether a blob is live. - The upload service drops off the upload path entirely after planning. It mints the tasks without contacting a node and never fans out to nodes at all. - Every node-side action is linked to the client's intent: the allocate's `cause` names the client's batch invocation. - Partial success is first-class. Blobs succeed and fail independently; delivery is idempotent and repeatable, so stragglers converge under the same batch. @@ -48,7 +47,7 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one ### The placement program -Today's `/blob/add` response already carries the upload as a program: the `/blob/allocate`, `/http/put` and `/blob/accept` invocations, promise-chained, each executed by a different party. This RFC keeps that contract and widens it. For each candidate placement of each blob, sprue pre-mints the full task chain: +Today's `/blob/add` response already carries the upload as a program: the `/blob/allocate`, `/http/put` and `/blob/accept` invocations, promise-chained, each executed by a different party. This RFC keeps that contract and widens it. For each blob, sprue pre-mints the full task chain: | Task | Executor | Chaining | | --- | --- | --- | @@ -57,26 +56,26 @@ Today's `/blob/add` response already carries the upload as a program: the `/blob | `/blob/accept` | the node (the agent relays the invocation) | `_put` awaits the put task | | `/blob/register` | sprue, at delivery | `accept` links the accept task | -The allocate and accept invocations are issued and signed by sprue with the provider as subject, under the provider's registration delegation — the same chain nodes validate today; sprue likewise authors the put task under the digest-derived key, and the sprue-subject register task. The agent adds no authority and can vary nothing, only execute or not execute. All four are built without nonces, so their task CIDs are deterministic and known to every party from the moment the response lands, including the register task, whose receipt the client can await from t=0. The allocate, put and accept tasks carry an explicit expiry equal to the batch's upload window (the 30-second invocation default would be useless here). The register task expires later, at the registration horizon: the window plus a delivery allowance for receipts to land. +The allocate and accept invocations are issued and signed by sprue with the provider as subject, under the provider's registration delegation — the same chain nodes validate today; sprue likewise authors the put task under the digest-derived key, and the sprue-subject register task. The agent adds no authority and can vary nothing, only execute or not execute. All four are built without nonces, so their task CIDs are deterministic and known to every party from the moment the response lands, including the register task, whose receipt the client can await from t=0. All four tasks carry an explicit expiry equal to the batch's upload window (the 30-second invocation default would be useless here); a chain registers until that expiry plus the protocol grace period — the registration horizon. -Because allocate's subject is the provider, the chain is minted per candidate: a blob with three candidates gets three parallel programs, of which the client executes one (or more, up to the replica count). Unexecuted programs cost nothing and expire. +One blob, one node, one program — exactly today's shape. A blob re-placed by `/blob/batch/extend` gets a second program at a different provider; unexecuted programs cost nothing and expire. One artifact in the placement is not an invocation: a **`/blob/reject` delegation**, audience the agent, its policy pinning the space and the digest so the agent can unwind only its own placement's parked state — content-addressed dedup means another tenant can hold a parked allocation for the same digest, and an unpinned space field would reach it. Reject unwinds the upload rather than advancing it, and that difference is the security line: the program's tasks are idempotent and produce only what the space announced, so they are safe to execute on delivery no matter who carries them, while reject carries destructive discretion over in-flight state and is therefore key-bound to the agent. It is safe against anything accepted: reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). -### Over-allocation and the race +### Sharding clients and the long tail -Placement returns more candidates than a blob needs. The agent allocates at its candidates, PUTs to as many as it likes, and stops when enough land — the race is client policy, from "primary first, spare on failure" to "start all, cancel at first." A candidate whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and is an instant winner. +Placement is one node per blob; over-provisioning belongs to the client's blob set. A client that erasure-codes or shards announces more piece-blobs than its code needs, drives the PUTs in parallel, and stops driving blobs once enough of its pieces land. Cancelling at enough is client arithmetic across blobs; the abandoned tail is parked state that expires node-side with no message, or is retired eagerly — abort the tail digests, then reject them at their nodes. A blob whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and lands instantly. -The agent then relays `/blob/accept` at winners, and MUST accept at no more than the blob's replica count of them: acceptance is the durable state, and registration will not pay for more. A retried accept that lands late anyway can still leave one extra acceptance; that converges rather than corrupts — the surplus node learns its status from its own delivery outcome and reclaims, readers who hit its stale location claim retry the registered node's claim, and the agent reconciles its index from the delivery outcomes. When relaying an accept, the agent MUST include the candidate's register invocation, put invocation and put receipt in the same container: those artifacts are what the node's own delivery needs, and the node refuses the accept without them. +The agent relays `/blob/accept` for the blobs it means to keep, and MUST include the placement's register invocation, put invocation and put receipt in the same container: those artifacts are what the node's own delivery needs, and the node refuses the accept without them. -Losing candidates need no message at all: their parked allocations, and any bytes from PUTs cancelled after completion, expire node-side. +A blob can hold acceptances at two providers only across `/blob/batch/extend` rounds — the agent re-placed after a presumed failure and the first accept landed anyway. That converges: the registration bound settles one provider, the other learns `Surplus` from its own delivery and reclaims, readers who hit the stale location claim retry the registered provider's claim, and the agent reconciles its index from the register receipts. ### Batch lifecycle -1. **Plan** — `/blob/batch/add`. Sprue selects candidate nodes per blob over its provider directory in one pass, mints the placement programs, and returns them. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. -2. **Allocate, race, accept** — one container of allocate invocations per candidate node; PUTs in parallel; cancel at enough; one container of accept invocations per winning node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. +1. **Plan** — `/blob/batch/add`. Sprue selects one node per blob over its provider directory in one pass, mints the placement programs, and returns them. Sprue contacts no node; blobs already registered in the space are returned as such, with no placement. +2. **Allocate, upload, accept** — one container of allocate invocations per node; PUTs in parallel; one container of accept invocations per node. The agent ends the node exchange holding node-signed accept receipts, location claims and PDP promises for every blob that landed. This is the client's completion point. 3. **Deliver** — the node SHOULD invoke `/ucan/conclude` on sprue for each accept it executes; the agent SHOULD deliver its collected accept receipts in one batched `/ucan/conclude`. Sprue verifies each delivered chain, executes the pre-minted `/blob/register` task idempotently, and answers with per-receipt outcomes linking its receipt. -4. **Repair the tail** — a blob whose candidates all failed gets fresh ones from `/blob/batch/extend` and its receipts are delivered later under the same batch. A batch the client walks away from is cancelled with `/blob/batch/abort`. +4. **Repair** — a blob whose node failed gets a replacement placement from `/blob/batch/extend` and its receipts are delivered later under the same batch. A batch the client walks away from is cancelled with `/blob/batch/abort`. ### Transport @@ -90,7 +89,7 @@ Batching travels as ucantone container contents: a single HTTP exchange carries * Audience: Sprue * Subject: The space -Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns candidate placements per blob, with the placement programs' invocations and the reject delegations carried in the response container alongside the provider registration delegations the node-facing invocations need as proofs. Candidates come only from nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. The result names one primary candidate per blob in `placements` and additional candidates in `spares`; how many spares to race and how many to hold in reserve is client policy. Digests already registered in the space are listed in `registered` and receive no placement. +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns one placement per blob, with the placement programs' invocations and the reject delegations carried in the response container alongside the provider registration delegations the node-facing invocations need as proofs. Placements go only to nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. Digests already registered in the space are listed in `registered` and receive no placement. The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. @@ -146,7 +145,7 @@ e.g. ```ipldsch type Placement struct { digest Bytes # multihash of the blob - provider String # DID of the candidate storage node + provider String # DID of the selected storage node allocate Link # the /blob/allocate invocation, in the response container put Link # the /http/put invocation accept Link # the /blob/accept invocation @@ -155,8 +154,7 @@ type Placement struct { } type BatchAddOK struct { - placements [Placement] # primary candidate per blob - spares [Placement] # additional candidates; race or hold in reserve, client's choice + placements [Placement] # one placement per blob registered [Bytes] # digests already registered in this space; no placement issued } ``` @@ -177,7 +175,6 @@ type Placement struct { type BatchAddOK struct { Placements []Placement `cborgen:"placements"` - Spares []Placement `cborgen:"spares"` Registered []multihash.Multihash `cborgen:"registered"` } ``` @@ -198,11 +195,11 @@ Sprue dispatches each delivered receipt whose `Ran` command has a registered con 1. verifies the accept receipt is signed by the provider DID that is the accept invocation's subject, with `Ran` matching and a success result — else `InvalidReceipt`; 2. verifies the accept and register invocations are **issued and signed by sprue itself**, and that the register invocation's `accept` field names the delivered accept task. Sprue verifying its own signatures is the placement check, and it requires no stored state — the register task exists only for placements sprue minted, which also keeps legacy sprue-signed accepts out of this path — failures return `UnknownPlacement`; 3. enforces the registration horizon: a register task that expired more than the protocol grace period ago MUST be refused with an `Expired` outcome. Within the horizon, freshness is not otherwise rechecked — the node-signed receipt attests that the accept validated, expiry included, at execution time; -4. enforces the registration bound: at most the replica count the register invocation names is registered per `(digest, space)`. The earliest delivered acceptances win, up to the bound; further acceptances of the same blob at other providers return a `Surplus` outcome and are not registered; +4. enforces the registration bound: one acceptance registers per `(digest, space)` — the first delivered wins, and a later acceptance of the same blob at another provider returns a `Surplus` outcome and is not registered; 5. checks the tombstones (below): a chain matching one returns a `Removed` outcome instead of registering; 6. registers the blob by executing the register task — issuing its receipt — debits the space's quota, and persists the delivered chain and the registration alongside it. -Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` tasks and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement was minted before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). `/blob/batch/abort` writes the same pending tombstones for a whole batch at once. Both checks compare the delivered artifacts — the register task's `cause` and issuance time — against the tombstone record alone, so sprue keeps no journal of in-flight uploads. +Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` task and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement was minted before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). `/blob/batch/abort` writes the same pending tombstones for a whole batch at once. Both checks compare the delivered artifacts — the register task's `cause` and issuance time — against the tombstone record alone, so sprue keeps no journal of in-flight uploads. Delivery is idempotent and unordered: the register task's CID is fixed at minting, so the node's conclusion and the agent's conclusion of the same accept converge on the identical task and one receipt, and a repeat delivery returns the same registration link. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else, and a `Surplus` outcome still carries the blob's registration links, so the deliverer learns the registered providers directly. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. @@ -290,9 +287,9 @@ type BatchConcludeOK struct { * Audience: Sprue * Subject: Sprue -The terminal task of the placement program, and a virtual task in the tradition of `/http/put`: no node serves it. Sprue mints it at `/blob/batch/add` alongside the other three tasks and executes it — issues its receipt — inside the `/blob/accept` conclusion handler when a delivery verifies. The receipt is the canonical, sprue-signed record that a blob is registered — the artifact billing, index reconciliation and disputes start from — and the registration time is the receipt's issue time. Sprue sets `replicas` from the space's placement policy at planning time; extend-minted register tasks carry the original batch's bound. Because the task CID is fixed at minting, the client can await this receipt from the moment the batch-add response lands. +The terminal task of the placement program, and a virtual task in the tradition of `/http/put`: no node serves it. Sprue mints it at `/blob/batch/add` alongside the other three tasks and executes it — issues its receipt — inside the `/blob/accept` conclusion handler when a delivery verifies. The receipt is the canonical, sprue-signed record that a blob is registered — the artifact billing, index reconciliation and disputes start from — and the registration time is the receipt's issue time. Because the task CID is fixed at minting, the client can await this receipt from the moment the batch-add response lands. -Exactly one receipt exists per registered acceptance, no matter who delivered first or how many times. Removing a registered blob retires its register tasks; the cause-keyed tombstone records them. +Exactly one receipt exists per registered blob, no matter who delivered first or how many times. Removing a registered blob retires its register task; the cause-keyed tombstone records it. Idempotent: a delivery that finds the registration already executed returns the existing receipt's link in its outcome. @@ -307,7 +304,6 @@ type RegisterArguments struct { provider String # DID of the storage node holding the registered acceptance cause Link # the /blob/batch/add task accept Link # the /blob/accept task whose delivery triggers registration - replicas Int # registration bound for this blob, per (digest, space) } ``` @@ -321,7 +317,6 @@ type RegisterArguments struct { Provider did.DID `cborgen:"provider"` Cause cid.Cid `cborgen:"cause"` Accept cid.Cid `cborgen:"accept"` - Replicas uint64 `cborgen:"replicas"` } ``` @@ -336,7 +331,7 @@ Successful registration returns a unit result (`{}`). The receipt's substance is * Audience: Sprue * Subject: The space -Requests replacement candidates for blobs whose existing candidates have failed. Each entry MAY exclude providers the client has already failed against; sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK`: fresh placement programs with fresh expiries and a fresh registration horizon for the re-placed blobs, whose quota holds replace the superseded candidates' outstanding ones, and whose tasks bind `cause` to the **original** batch-add task, so the whole batch delivers under one cause. Extend MUST NOT issue a placement for a digest carrying a live cause-keyed tombstone under this cause, and extending an aborted batch MUST fail with the named error `BatchAborted`. +Requests a replacement placement for blobs whose node has failed. Each entry MAY exclude providers the client has already failed against — exclusions are client-supplied and SHOULD be cumulative across repair rounds, so a provider that failed in any earlier round stays excluded — and sprue MUST NOT place an entry on an excluded provider. The result is a `BatchAddOK`: fresh placement programs with fresh expiries and a fresh registration horizon for the re-placed blobs, whose quota holds replace the superseded placements' outstanding ones, and whose tasks bind `cause` to the **original** batch-add task, so the whole batch delivers under one cause. Extend MUST NOT issue a placement for a digest carrying a live cause-keyed tombstone under this cause, and extending an aborted batch MUST fail with the named error `BatchAborted`. Idempotent: extending a digest that has since registered simply returns no placement for it, listed in `registered`. @@ -403,11 +398,11 @@ e.g. * Audience: Sprue * Subject: The space -Cancels a batch's outstanding placements: the client renounces every placement of the batch that has not registered. Sprue releases the batch's remaining quota holds and writes the pending tombstones immediately, so chains from the abandoned placements return `Removed` instead of registering — the abort is, in effect, revocation of the outstanding placement programs. Blobs already registered are untouched and listed in the result; they exit via `/blob/remove`. +Cancels a batch's outstanding placements — all of them, or only the digests named in `blobs`: the client renounces the placements that have not registered. A sharding client aborts its over-provisioned tail this way while the rest of the batch stays live. Sprue releases the batch's remaining quota holds and writes the pending tombstones immediately, so chains from the abandoned placements return `Removed` instead of registering — the abort is, in effect, revocation of the outstanding placement programs. Blobs already registered are untouched and listed in the result; they exit via `/blob/remove`. Abort is the upload-service half of cancellation. For node-side cleanup the client invokes `/blob/reject` under the placements' reject delegations, one container per node it touched, and SHOULD abort first so the tombstones are in place before any in-flight delivery races the cleanup. Acceptances that already executed cannot be rejected (`BlobAccepted`); they resolve through the delivery loop — the node concludes, receives `Removed`, and reclaims. -Abort terminates the cause: a later `/blob/batch/extend` under it fails with `BatchAborted`, so a cancelled batch cannot mint post-tombstone placements that slip past its own cancellation. +A whole-batch abort terminates the cause: a later `/blob/batch/extend` under it fails with `BatchAborted`, so a cancelled batch cannot mint post-tombstone placements that slip past its own cancellation. A digest-scoped abort terminates only those digests — extend refuses them and the rest of the batch continues. Idempotent: aborting an unknown, already-aborted, or fully-registered batch MUST succeed. @@ -417,7 +412,8 @@ Idempotent: aborting an unknown, already-aborted, or fully-registered batch MUST ```ipldsch type BatchAbortArguments struct { - cause Link # the /blob/batch/add task being cancelled + cause Link # the /blob/batch/add task being cancelled + blobs optional [Bytes] # limit the abort to these digests; absent, the whole batch } ``` @@ -426,7 +422,8 @@ type BatchAbortArguments struct { ```go type BatchAbortArguments struct { - Cause cid.Cid `cborgen:"cause"` + Cause cid.Cid `cborgen:"cause"` + Blobs []multihash.Multihash `cborgen:"blobs,omitempty"` } ``` @@ -482,58 +479,58 @@ The client's completion does not wait for any of this. Resolvability comes from ## Failure handling and idempotency -- A failed or stalled PUT races another candidate; no sprue call. An exhausted candidate set costs one `/blob/batch/extend`. -- A refused accept is visible to the agent directly in the node's response; the blob goes back through the race or through extend. +- A sharding client outruns a slow node by finishing other pieces; a blob that must land and cannot costs one `/blob/batch/extend`. +- A refused accept is visible to the agent directly in the node's response; the blob goes back through extend. - An `Expired` outcome means the batch's registration horizon lapsed undelivered; the blob re-uploads under a fresh batch. - The agent SHOULD reconcile its index against the `/blob/register` receipts linked from its delivery outcomes: they name the registered providers, whose location claims are the durable ones. -- Allocations that never receive bytes, and bytes from cancelled or losing PUTs, expire on the node's existing allocation-expiry hygiene, with no message required. An abandoned batch is cancelled sooner with `/blob/batch/abort` plus reject. +- Allocations that never receive bytes, and bytes from PUTs the client cancelled or abandoned, expire on the node's existing allocation-expiry hygiene, with no message required. An abandoned batch or tail is cancelled sooner with `/blob/batch/abort` plus reject. - Every capability in this RFC is idempotent: re-adding, re-delivering, re-extending and re-aborting converge on the same state, so the retry story for the whole protocol is "re-invoke". ## Example flow -An object of M blobs raced across N candidate nodes: +An object of M blobs placed across N nodes: | # | Leg | What moves | |---|-----|------------| | 1 | agent → sprue | one `/blob/batch/add`: M digests and sizes | -| 2 | sprue → agent | `BatchAddOK`: candidate placement programs in the container | -| 3 | agent → each node, parallel | one container relaying that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — warm, instant winner) | -| 4 | agent → nodes, parallel | HTTP PUTs to chosen candidates; cancel at enough; sign a put receipt per winner | -| 5 | agent → each winning node, parallel | one container relaying that node's `/blob/accept` invocations (register and put invocations and put receipts included); receipts carry location claims and PDP promises — **the client is done** | +| 2 | sprue → agent | `BatchAddOK`: one placement program per blob in the container | +| 3 | agent → each node, parallel | one container relaying that node's `/blob/allocate` invocations; receipts carry upload addresses (or none — the node already holds the bytes) | +| 4 | agent → nodes, parallel | HTTP PUTs in parallel; sign a put receipt per landed blob (a sharding client stops at enough pieces) | +| 5 | agent → each node, parallel | one container relaying that node's `/blob/accept` invocations (register and put invocations and put receipts included); receipts carry location claims and PDP promises — **the client is done** | | 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | | 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response links a sprue-signed `/blob/register` receipt per registered acceptance | -A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs that failed everywhere, repeats legs 3–5 against the fresh candidates, and the late receipts ride the same delivery path under the original cause. A batch the client abandons is cancelled with `/blob/batch/abort` followed by `/blob/reject` at the touched nodes. +A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs whose nodes failed, repeats legs 3–5 against the fresh placements, and the late receipts ride the same delivery path under the original cause. A batch the client abandons is cancelled with `/blob/batch/abort` followed by `/blob/reject` at the touched nodes. Synchronous round trips: one to sprue per batch, two per node — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 41 synchronous control exchanges, all parallel across nodes, plus at most 21 asynchronous receipt deliveries off the completion path — instead of ~500, serial per blob. Byte PUTs are excluded from both counts. ## Security considerations - **The program is fixed.** Every node-facing artifact is a complete sprue-signed invocation: all arguments set, nothing for the holder to vary. The agent's entire latitude is which pre-authorized programs to execute. -- **Bearer execution is bounded by the announcement.** The invocations execute on delivery, so a holder of a leaked response container can advance the program — at worst force-completing an announced upload of content it possesses, billing the space for blobs and sizes the space itself named, inside the expiry window. It cannot divert the program to other digests, sizes, spaces, or non-candidate nodes; within a blob's candidate set a faster delivery can pick a different winner than the agent chose, which converges through `Surplus` and reclaim like any late acceptance. `/blob/batch/abort`'s tombstones close even the forced-completion sliver for a batch the client cancels. The one destructive right, reject, is key-bound to the agent precisely because a bearer form of it would let an interceptor destroy in-flight uploads. +- **Bearer execution is bounded by the announcement.** The invocations execute on delivery, so a holder of a leaked response container can advance the program — at worst force-completing an announced upload of content it possesses, billing the space for blobs and sizes the space itself named, inside the expiry window. It cannot divert the program to other digests, sizes, spaces, or nodes; where an extend has left a blob with programs at two providers, a faster delivery can settle the one the agent abandoned, which converges through `Surplus` and reclaim like any late acceptance. `/blob/batch/abort`'s tombstones close the forced-completion sliver for placements the client renounces, whole batches or over-provisioned tails; a tail the client neither aborts nor rejects stays force-completable until its placements expire. The one destructive right, reject, is key-bound to the agent precisely because a bearer form of it would let an interceptor destroy in-flight uploads. - **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt for an accept invocation sprue itself signed, named by a register task sprue itself signed, inside the horizon. The conclude issuer is irrelevant by design. Registration produces evidence in turn: the sprue-signed `/blob/register` receipt is the record billing and disputes start from. -- **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that over-accepts gains nothing durable and costs the nodes storage until reclaim — and, on nodes that defer PDP enqueue to the registration signal, no proof obligations. +- **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that ends up with a second acceptance — possible only across extend rounds — gains nothing durable and costs the nodes storage until reclaim — and, on nodes that defer PDP enqueue to the registration signal, no proof obligations. - **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and nothing rests on it; the node-signed accept receipt is the artifact registration relies on. - **Replay.** Invocation task CIDs are deterministic and their effects idempotent, keyed `(digest, space)`; delivery, registration and reclaim all converge under repeats; the horizon refuses stale chains outright, and the tombstones stop a replayed delivery from resurrecting a removed registration in either race direction. - **Resource bounds.** A batch amplifies work: one invocation makes sprue run placement and mint programs for M blobs, and one conclude makes it verify M chains. Sprue SHOULD enforce configurable caps on blobs per batch and receipts per conclude. Quota SHOULD be taken as a hold with a TTL equal to the registration horizon, released on expiry or abort and reconciled at registration; registration inside the horizon debits unconditionally, so the over-commit exposure is bounded to chains delivered in the grace period after their hold released — an outright debit at add would leak on abandoned batches, and a bare check would over-commit across parallel batches. ## Limits -The cbor-gen list cap (8,192 elements) bounds every list in these schemas. A placement is five signed artifacts — four invocations and a delegation — a few hundred bytes each, so the batch-add response runs to a couple of megabytes per thousand candidates; batch and candidate caps size accordingly. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. A placement is five signed artifacts — four invocations and a delegation — a few hundred bytes each, so the batch-add response runs to a couple of megabytes per thousand blobs; the batch cap sizes accordingly. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. ## Compatibility - **Storage nodes**: the verbs, argument types, results and issuer are unchanged — nodes validate today's exact chain; only the courier differs. Outbound conclusion has an existing precedent: piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. -- **`/blob/add` and the [blob protocol spec](https://github.com/fil-forge/ucan-protocol-specs/blob/main/blob.md)**: the single-blob flow remains, unchanged, for callers that cannot know digests up front, and the spec's response-container contract generalizes rather than changes — the container still carries the upload's promise-chained task invocations; there are simply more of them, plus the explicit terminal register task. A streaming client MAY batch per flush window — announce the blobs whose digests it has, race, accept, deliver, repeat. +- **`/blob/add` and the [blob protocol spec](https://github.com/fil-forge/ucan-protocol-specs/blob/main/blob.md)**: the single-blob flow remains, unchanged, for callers that cannot know digests up front, and the spec's response-container contract generalizes rather than changes — the container still carries the upload's promise-chained task invocations; there are simply more of them, plus the explicit terminal register task. A streaming client MAY batch per flush window — announce the blobs whose digests it has, upload, accept, deliver, repeat. - **`/ucan/conclude`**: the plural field is additive, and the `/blob/accept` conclusion handler is a new entry in the existing dispatch map; existing single-receipt clients and the legacy `/http/put` handler are untouched. -- **The [blob-removal RFC](./2026-07-forge-blob-removal.md)** is amended in three stated places. Its receipt-chain routing walk (the registration's cause, then the add receipt's `site`, then the accept task) does not exist for batch registrations — `BatchAddOK` has no `site` — so for batch-registered blobs routing MUST start from the `/blob/register` receipts, which name the providers and link the accept tasks directly, and `/blob/remove` fans `/blob/release` to every registered acceptance's provider. Its remove handler gains duties: the pending tombstone on removal of an unregistered digest, and per-`(digest, space)` serialization with registration. And its lifecycle gains the reclaim exit for never-registered acceptances, as specified in [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs). `/blob/abort` cannot route for batch blobs (sprue never learns which candidates the client executed until delivery); `/blob/batch/abort` plus the reject delegations replace it at batch granularity, with node-side expiry as the passive backstop. +- **The [blob-removal RFC](./2026-07-forge-blob-removal.md)** is amended in three stated places. Its receipt-chain routing walk (the registration's cause, then the add receipt's `site`, then the accept task) does not exist for batch registrations — `BatchAddOK` has no `site` — so for batch-registered blobs routing MUST start from the `/blob/register` receipts, which name the providers and link the accept tasks directly, and `/blob/remove` sends `/blob/release` to the registered provider. Its remove handler gains duties: the pending tombstone on removal of an unregistered digest, and per-`(digest, space)` serialization with registration. And its lifecycle gains the reclaim exit for never-registered acceptances, as specified in [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs). `/blob/abort` cannot route for batch blobs (sprue never learns which placements the client executed until delivery); `/blob/batch/abort` plus the reject delegations replace it at batch granularity, with node-side expiry as the passive backstop. - **Read-on-write**: resolvability is preserved at its source — the node publishes its location claim synchronously inside accept, before the agent ever holds the receipt. Registration-dependent behavior converges with delivery. Index and upload registration (`/index/add`, `/upload/add`) follow the accept receipts exactly as today, and are out of scope here. ## Alternatives considered ### Policy-bound delegations instead of pre-minted invocations -The batch-add response could carry authority instead of tasks: sprue re-delegates `/blob/allocate` and `/blob/accept` to the agent — subject the provider, policy pinning space, digest, size and cause — and the agent authors the node-facing invocations itself. The two shapes are close: the artifact count per candidate is similar, the delivery, horizon, tombstone and advertisement machinery is identical, and nodes validate a provider-rooted chain either way. +The batch-add response could carry authority instead of tasks: sprue re-delegates `/blob/allocate` and `/blob/accept` to the agent — subject the provider, policy pinning space, digest, size and cause — and the agent authors the node-facing invocations itself. The two shapes are close: the artifact count per placement is similar, the delivery, horizon, tombstone and advertisement machinery is identical, and nodes validate a provider-rooted chain either way. What the delegation shape buys: the response container is key-bound, so an intercepted copy is inert without the agent's key, where pre-minted invocations execute on delivery (bounded to force-completing the announced upload, as the security section states); the agent's own signature appears on the node-facing chain, rooting every node action in the customer's intent chain rather than in sprue's authorship; and very large batches can compact to one delegation pair per provider with an `or` policy over the digest set, which invocations cannot imitate. @@ -541,21 +538,22 @@ What it costs, and why this RFC chose invocations: a policy pins only the fields ## Future work +- **Placement-level replication.** Registering one blob at several providers would reintroduce a replica count, carried by the register task so the registration bound stays readable off the delivered artifact. Out of scope here; the bound in this RFC is one. - **Fold the node legs into the PUT.** The end state carries the placement program in the PUT request header and returns the accept receipt, location claim and PDP promise in the response header, collapsing the per-node exchanges to zero and using `Expect: 100-continue` to recover the warm-path signal. That change touches the node and deserves its own RFC; nothing here precludes it — the program and the delivery model are the same. ## Open questions -- The values of the upload window (the upload tasks' expiry), the delivery allowance that sets the registration horizon (the register task's expiry), and the grace period beyond it; the window wants to be generous, on the order of hours. +- The values of the upload window (the tasks' expiry) and the grace period that extends it into the registration horizon; the window wants to be generous, on the order of hours. - The mechanics of the batch-support advertisement at registration: a capability proven in the registration container, alongside the four commands required today, or a flag on the provider record. - Whether the agent awaits its conclude response — and with it the `/blob/register` receipt — before the S3 200, or fires it after; this RFC leans to the latter since resolvability does not depend on it. -- Default and maximum sizes for the batch, conclude and candidate-set caps. +- Default and maximum sizes for the batch and conclude caps. ## Evaluation criteria - Synchronous control exchanges for an M-blob object drop from ~5M to 1 + 2N, measured on ingot's sharded-object upload, with zero polling. - End-to-end upload latency for a multi-blob object improves by at least the removed poll, conclusion wait and serialization overhead, measured against the current guppy/ingot path. - No growth in stranded state: accepted-but-unregistered blobs converge to zero — registered, or reclaimed with a visible `Expired`/`Surplus`/`Removed` outcome — under fault injection: node crash before conclude, agent crash before conclude, sprue restart, `/blob/remove` racing a late delivery in both directions, abort racing an in-flight delivery, and delivery after the horizon. -- Exactly one `/blob/register` receipt exists per registered acceptance under concurrent node and agent delivery. +- Exactly one `/blob/register` receipt exists per registered blob under concurrent node and agent delivery. - The legacy single-blob flow is byte-identical on the wire before and after deployment. ## References From 983842b6a1e523b58245630205225900146242cf Mon Sep 17 00:00:00 2001 From: hannahhoward Date: Wed, 9 Sep 2026 08:22:54 -0700 Subject: [PATCH 7/7] rfc: multi-blob add: abort forwards rejects; placement is pure program MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drops the /blob/reject delegation — the last non-invocation artifact in the placement. Node-side cleanup becomes sprue's translation, exactly as /blob/abort maps to /blob/reject in the removal RFC: batch abort writes the tombstones then forwards rejects to every renounced placement's provider, best-effort, routed from the stored batch-add receipt. The response container now holds only bearer-safe tasks; nothing a holder can do is destructive. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UycQw45qWtuVKmvrzZVEKa --- rfcs/2026-09-multi-blob-add.md | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/rfcs/2026-09-multi-blob-add.md b/rfcs/2026-09-multi-blob-add.md index 516ff30..f43bb5e 100644 --- a/rfcs/2026-09-multi-blob-add.md +++ b/rfcs/2026-09-multi-blob-add.md @@ -25,10 +25,10 @@ Cost for an M-blob object: one synchronous round trip to the upload service, one - One synchronous client↔upload-service exchange per batch, independent of blob count, plus one `/blob/batch/extend` exchange per repair round. - No polling and no conclusion wait: the agent holds every receipt it needs — accept, location claim, PDP promise — directly from the nodes, at the end of the node exchange. -- The response container carries the upload as executable tasks: each blob's placement is a complete, sprue-signed program, every argument fixed, nothing for a holder to vary. The one right outside the program, early unwind via `/blob/reject`, travels as a key-bound delegation. +- The response container carries the upload as executable tasks: each blob's placement is a complete, sprue-signed program, every argument fixed, nothing for a holder to vary — and nothing destructive. Every unwind flows through the upload service under the removal RFC's existing verbs. - `/ucan/conclude` remains an intentionally open delivery mechanism; the upload service trusts what the delivered chain proves and nothing about who delivered it. - Reporting duty sits with the party that bears the loss: the node concludes its own accepts, and the agent's delivery is the second, independent path. -- The upload service drops off the upload path entirely after planning. It mints the tasks without contacting a node and never fans out to nodes at all. +- The upload service drops off the upload path after planning. It mints the tasks without contacting a node; only cancellation and removal fan out, best-effort, as the removal RFC already does. - Every node-side action is linked to the client's intent: the allocate's `cause` names the client's batch invocation. - Partial success is first-class. Blobs succeed and fail independently; delivery is idempotent and repeatable, so stragglers converge under the same batch. - The receipt chain keeps today's shape — `add → allocate → put → accept` — and gains a sprue-signed terminal link: registration itself becomes a chain-visible `/blob/register` receipt. @@ -60,11 +60,11 @@ The allocate and accept invocations are issued and signed by sprue with the prov One blob, one node, one program — exactly today's shape. A blob re-placed by `/blob/batch/extend` gets a second program at a different provider; unexecuted programs cost nothing and expire. -One artifact in the placement is not an invocation: a **`/blob/reject` delegation**, audience the agent, its policy pinning the space and the digest so the agent can unwind only its own placement's parked state — content-addressed dedup means another tenant can hold a parked allocation for the same digest, and an unpinned space field would reach it. Reject unwinds the upload rather than advancing it, and that difference is the security line: the program's tasks are idempotent and produce only what the space announced, so they are safe to execute on delivery no matter who carries them, while reject carries destructive discretion over in-flight state and is therefore key-bound to the agent. It is safe against anything accepted: reject refuses blobs the invoking space has accepted (`BlobAccepted`, per the [blob-removal RFC](./2026-07-forge-blob-removal.md)). +The placement carries nothing but the program, and the distinction is the security line: the program's tasks are idempotent and produce only what the space announced, so they are safe to execute on delivery no matter who carries them, while anything destructive — unwinding parked state, releasing accepted state — is a space-authorized verb the client invokes on sprue, which translates it to the node under the [blob-removal RFC](./2026-07-forge-blob-removal.md)'s existing scheme. The client never holds a node-facing artifact whose misuse could hurt anyone. ### Sharding clients and the long tail -Placement is one node per blob; over-provisioning belongs to the client's blob set. A client that erasure-codes or shards announces more piece-blobs than its code needs, drives the PUTs in parallel, and stops driving blobs once enough of its pieces land. Cancelling at enough is client arithmetic across blobs; the abandoned tail is parked state that expires node-side with no message, or is retired eagerly — abort the tail digests, then reject them at their nodes. A blob whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and lands instantly. +Placement is one node per blob; over-provisioning belongs to the client's blob set. A client that erasure-codes or shards announces more piece-blobs than its code needs, drives the PUTs in parallel, and stops driving blobs once enough of its pieces land. Cancelling at enough is client arithmetic across blobs; the abandoned tail is parked state that expires node-side with no message, or is retired eagerly with a digest-scoped `/blob/batch/abort`, which forwards the rejects. A blob whose allocate receipt carries no upload address already holds the bytes (the warm path, unchanged) and lands instantly. The agent relays `/blob/accept` for the blobs it means to keep, and MUST include the placement's register invocation, put invocation and put receipt in the same container: those artifacts are what the node's own delivery needs, and the node refuses the accept without them. @@ -89,7 +89,7 @@ Batching travels as ucantone container contents: a single HTTP exchange carries * Audience: Sprue * Subject: The space -Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns one placement per blob, with the placement programs' invocations and the reject delegations carried in the response container alongside the provider registration delegations the node-facing invocations need as proofs. Placements go only to nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. Digests already registered in the space are listed in `registered` and receive no placement. +Announces a batch of blobs for upload. Sprue verifies the caller's space authority and quota exactly as `/blob/add` does, plans placement for the whole batch in one selection pass, and returns one placement per blob, with the placement programs' invocations carried in the response container alongside the provider registration delegations the node-facing invocations need as proofs. Placements go only to nodes that advertised batch support at registration (see [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs)); sprue MUST NOT hand batch placements to a node that has not. Digests already registered in the space are listed in `registered` and receive no placement. The upload address is not part of the result: it comes from the node itself, in the `/blob/allocate` receipt, as today. @@ -150,7 +150,6 @@ type Placement struct { put Link # the /http/put invocation accept Link # the /blob/accept invocation register Link # the /blob/register invocation, executed by sprue at delivery - reject Link # the /blob/reject delegation, for early unwind } type BatchAddOK struct { @@ -170,7 +169,6 @@ type Placement struct { Put cid.Cid `cborgen:"put"` Accept cid.Cid `cborgen:"accept"` Register cid.Cid `cborgen:"register"` - Reject cid.Cid `cborgen:"reject"` } type BatchAddOK struct { @@ -199,7 +197,7 @@ Sprue dispatches each delivered receipt whose `Ran` command has a registered con 5. checks the tombstones (below): a chain matching one returns a `Removed` outcome instead of registering; 6. registers the blob by executing the register task — issuing its receipt — debits the space's quota, and persists the delivered chain and the registration alongside it. -Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` task and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement was minted before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). `/blob/batch/abort` writes the same pending tombstones for a whole batch at once. Both checks compare the delivered artifacts — the register task's `cause` and issuance time — against the tombstone record alone, so sprue keeps no journal of in-flight uploads. +Registration and removal MUST serialize per `(digest, space)`, and two tombstones make their races come out right in both directions. Removing a **registered** blob retires its `/blob/register` task and tombstones its `(digest, space, cause)`: a redelivered chain for that cause returns `Removed`, while a fresh batch's chain — new cause — registers. Removing a digest with **no registration yet** writes a pending tombstone stamped with the removal time, held for the length of the registration horizon: it blocks chains whose placement was minted before the stamp (the batch the client walked away from) and passes chains minted after it (a re-upload). `/blob/batch/abort` writes the same pending tombstones for a whole batch at once. Both checks compare the delivered artifacts — the register task's `cause` and issuance time — against the tombstone record alone, so verifying a delivery needs no record of in-flight uploads. Delivery is idempotent and unordered: the register task's CID is fixed at minting, so the node's conclusion and the agent's conclusion of the same accept converge on the identical task and one receipt, and a repeat delivery returns the same registration link. A delivery failing any check lands in the outcome list with a named reason and blocks nothing else, and a `Surplus` outcome still carries the blob's registration links, so the deliverer learns the registered providers directly. The legacy `/http/put` conclusion handler is unchanged and continues to serve the single-blob flow. @@ -400,7 +398,7 @@ e.g. Cancels a batch's outstanding placements — all of them, or only the digests named in `blobs`: the client renounces the placements that have not registered. A sharding client aborts its over-provisioned tail this way while the rest of the batch stays live. Sprue releases the batch's remaining quota holds and writes the pending tombstones immediately, so chains from the abandoned placements return `Removed` instead of registering — the abort is, in effect, revocation of the outstanding placement programs. Blobs already registered are untouched and listed in the result; they exit via `/blob/remove`. -Abort is the upload-service half of cancellation. For node-side cleanup the client invokes `/blob/reject` under the placements' reject delegations, one container per node it touched, and SHOULD abort first so the tombstones are in place before any in-flight delivery races the cleanup. Acceptances that already executed cannot be rejected (`BlobAccepted`); they resolve through the delivery loop — the node concludes, receives `Removed`, and reclaims. +Node-side cleanup is sprue's translation, exactly as `/blob/abort` translates to `/blob/reject` today: after writing the tombstones, sprue forwards a `/blob/reject` to every renounced placement's provider under its registration delegation, resolving the providers from its stored batch-add receipt. Forwarding is best-effort — the node handler is idempotent, a placement whose allocate never executed rejects as an unknown blob, and node-side expiry reaps whatever a missed forward leaves — and the tombstones land first, so no in-flight delivery races the cleanup. Acceptances that already executed cannot be rejected (`BlobAccepted`); they resolve through the delivery loop — the node concludes, receives `Removed`, and reclaims. A whole-batch abort terminates the cause: a later `/blob/batch/extend` under it fails with `BatchAborted`, so a cancelled batch cannot mint post-tombstone placements that slip past its own cancellation. A digest-scoped abort terminates only those digests — extend refuses them and the rest of the batch continues. @@ -483,7 +481,7 @@ The client's completion does not wait for any of this. Resolvability comes from - A refused accept is visible to the agent directly in the node's response; the blob goes back through extend. - An `Expired` outcome means the batch's registration horizon lapsed undelivered; the blob re-uploads under a fresh batch. - The agent SHOULD reconcile its index against the `/blob/register` receipts linked from its delivery outcomes: they name the registered providers, whose location claims are the durable ones. -- Allocations that never receive bytes, and bytes from PUTs the client cancelled or abandoned, expire on the node's existing allocation-expiry hygiene, with no message required. An abandoned batch or tail is cancelled sooner with `/blob/batch/abort` plus reject. +- Allocations that never receive bytes, and bytes from PUTs the client cancelled or abandoned, expire on the node's existing allocation-expiry hygiene, with no message required. An abandoned batch or tail is cancelled sooner with `/blob/batch/abort`, which forwards the rejects. - Every capability in this RFC is idempotent: re-adding, re-delivering, re-extending and re-aborting converge on the same state, so the retry story for the whole protocol is "re-invoke". ## Example flow @@ -500,14 +498,14 @@ An object of M blobs placed across N nodes: | 6 | node → sprue, async | each node SHOULD `/ucan/conclude` the accepts it executed | | 7 | agent → sprue, async | the agent SHOULD deliver its collected receipts in one batched `/ucan/conclude`; the response links a sprue-signed `/blob/register` receipt per registered acceptance | -A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs whose nodes failed, repeats legs 3–5 against the fresh placements, and the late receipts ride the same delivery path under the original cause. A batch the client abandons is cancelled with `/blob/batch/abort` followed by `/blob/reject` at the touched nodes. +A repair round inserts between legs 5 and 7: the agent calls `/blob/batch/extend` for the blobs whose nodes failed, repeats legs 3–5 against the fresh placements, and the late receipts ride the same delivery path under the original cause. A batch the client abandons is cancelled with one `/blob/batch/abort`; sprue forwards the rejects. Synchronous round trips: one to sprue per batch, two per node — against three exchanges with sprue per blob (`/blob/add`, `/ucan/conclude`, at least one poll GET) and two sprue→node invocations per blob today. For a 100-blob object on 20 nodes: at most 41 synchronous control exchanges, all parallel across nodes, plus at most 21 asynchronous receipt deliveries off the completion path — instead of ~500, serial per blob. Byte PUTs are excluded from both counts. ## Security considerations - **The program is fixed.** Every node-facing artifact is a complete sprue-signed invocation: all arguments set, nothing for the holder to vary. The agent's entire latitude is which pre-authorized programs to execute. -- **Bearer execution is bounded by the announcement.** The invocations execute on delivery, so a holder of a leaked response container can advance the program — at worst force-completing an announced upload of content it possesses, billing the space for blobs and sizes the space itself named, inside the expiry window. It cannot divert the program to other digests, sizes, spaces, or nodes; where an extend has left a blob with programs at two providers, a faster delivery can settle the one the agent abandoned, which converges through `Surplus` and reclaim like any late acceptance. `/blob/batch/abort`'s tombstones close the forced-completion sliver for placements the client renounces, whole batches or over-provisioned tails; a tail the client neither aborts nor rejects stays force-completable until its placements expire. The one destructive right, reject, is key-bound to the agent precisely because a bearer form of it would let an interceptor destroy in-flight uploads. +- **Bearer execution is bounded by the announcement.** The invocations execute on delivery, so a holder of a leaked response container can advance the program — at worst force-completing an announced upload of content it possesses, billing the space for blobs and sizes the space itself named, inside the expiry window. It cannot divert the program to other digests, sizes, spaces, or nodes; where an extend has left a blob with programs at two providers, a faster delivery can settle the one the agent abandoned, which converges through `Surplus` and reclaim like any late acceptance. `/blob/batch/abort`'s tombstones close the forced-completion sliver for placements the client renounces, whole batches or over-provisioned tails; a tail the client never aborts stays force-completable until its placements expire. Nothing in the container is destructive: an interceptor can only make announced uploads happen, never unwind anyone's state. - **Registration requires evidence.** No delivered artifact moves state unless the chain proves it: a node-signed accept receipt for an accept invocation sprue itself signed, named by a register task sprue itself signed, inside the horizon. The conclude issuer is irrelevant by design. Registration produces evidence in turn: the sprue-signed `/blob/register` receipt is the record billing and disputes start from. - **Surplus acceptance is bounded twice.** The registration bound caps what sprue registers per `(digest, space)`; the reclaim deadline — a commitment of every batch-advertising node — returns unregistered acceptances to reclaimable state. An agent that ends up with a second acceptance — possible only across extend rounds — gains nothing durable and costs the nodes storage until reclaim — and, on nodes that defer PDP enqueue to the registration signal, no proof obligations. - **Evidence strength.** The put receipt is a client assertion — its digest-derived key was always mintable by any holder of the digest — and nothing rests on it; the node-signed accept receipt is the artifact registration relies on. @@ -516,14 +514,14 @@ Synchronous round trips: one to sprue per batch, two per node — against three ## Limits -The cbor-gen list cap (8,192 elements) bounds every list in these schemas. A placement is five signed artifacts — four invocations and a delegation — a few hundred bytes each, so the batch-add response runs to a couple of megabytes per thousand blobs; the batch cap sizes accordingly. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. +The cbor-gen list cap (8,192 elements) bounds every list in these schemas. A placement is four signed invocations, a few hundred bytes each, so the batch-add response runs to a couple of megabytes per thousand blobs; the batch cap sizes accordingly. `MaxBlobSize` (256 MiB) is unchanged, enforced node-side at allocate as today. ## Compatibility - **Storage nodes**: the verbs, argument types, results and issuer are unchanged — nodes validate today's exact chain; only the courier differs. Outbound conclusion has an existing precedent: piri already delivers replica-transfer receipts via `/ucan/conclude`. What is new is opt-in: a node advertises batch support at registration, taking on the delivery and reclaim behaviors, and sprue places batches only on nodes that have. Nodes that never advertise keep serving the legacy flow exactly as today and see no batch traffic, so the delivery model's guarantees never depend on a node that hasn't signed up for them. - **`/blob/add` and the [blob protocol spec](https://github.com/fil-forge/ucan-protocol-specs/blob/main/blob.md)**: the single-blob flow remains, unchanged, for callers that cannot know digests up front, and the spec's response-container contract generalizes rather than changes — the container still carries the upload's promise-chained task invocations; there are simply more of them, plus the explicit terminal register task. A streaming client MAY batch per flush window — announce the blobs whose digests it has, upload, accept, deliver, repeat. - **`/ucan/conclude`**: the plural field is additive, and the `/blob/accept` conclusion handler is a new entry in the existing dispatch map; existing single-receipt clients and the legacy `/http/put` handler are untouched. -- **The [blob-removal RFC](./2026-07-forge-blob-removal.md)** is amended in three stated places. Its receipt-chain routing walk (the registration's cause, then the add receipt's `site`, then the accept task) does not exist for batch registrations — `BatchAddOK` has no `site` — so for batch-registered blobs routing MUST start from the `/blob/register` receipts, which name the providers and link the accept tasks directly, and `/blob/remove` sends `/blob/release` to the registered provider. Its remove handler gains duties: the pending tombstone on removal of an unregistered digest, and per-`(digest, space)` serialization with registration. And its lifecycle gains the reclaim exit for never-registered acceptances, as specified in [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs). `/blob/abort` cannot route for batch blobs (sprue never learns which placements the client executed until delivery); `/blob/batch/abort` plus the reject delegations replace it at batch granularity, with node-side expiry as the passive backstop. +- **The [blob-removal RFC](./2026-07-forge-blob-removal.md)** is amended in three stated places. Its receipt-chain routing walk (the registration's cause, then the add receipt's `site`, then the accept task) does not exist for batch registrations — `BatchAddOK` has no `site` — so for batch-registered blobs routing MUST start from the `/blob/register` receipts, which name the providers and link the accept tasks directly, and `/blob/remove` sends `/blob/release` to the registered provider. Its remove handler gains duties: the pending tombstone on removal of an unregistered digest, and per-`(digest, space)` serialization with registration. And its lifecycle gains the reclaim exit for never-registered acceptances, as specified in [the node legs](#the-node-legs-bloballocate-and-blobaccept-unchanged-verbs). `/blob/batch/abort` is its batch form: the same client-verb-to-`/blob/reject` translation, routed from the stored batch-add receipt instead of a per-blob cause, with node-side expiry as the passive backstop. - **Read-on-write**: resolvability is preserved at its source — the node publishes its location claim synchronously inside accept, before the agent ever holds the receipt. Registration-dependent behavior converges with delivery. Index and upload registration (`/index/add`, `/upload/add`) follow the accept receipts exactly as today, and are out of scope here. ## Alternatives considered