Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/native-conditional-operations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"files-sdk": minor
---

Add provider-native conditional create, replace, exact-read, delete, and copy operations to the existing plugin, hook, retry, prefix, read-only, and receipt pipeline. AWS S3 implements the initial atomic primitives; unsupported adapters, filesystem, custom S3 endpoints, R2, bulk, and multipart paths fail closed.

The S3 adapter exposes the primitives only for canonical AWS — no `endpoint` and no `AWS_ENDPOINT_URL_S3` / `AWS_ENDPOINT_URL` redirect — with a new `conditional` option to override that in either direction, and verifies per request that the resolved hostname is AWS (so a shared-config `endpoint_url` fails closed too) and that the installed `@aws-sdk/client-s3` serialized every predicate header (conditional copy needs 3.919.0+; the `@aws-sdk/client-s3` peer range moves from `^3.700.0` to `^3.1079.0`). The CLI gains `--if-match` / `--if-none-match` / `--dest-if-match` on `upload`, `download`, `delete`, and `copy`, and the MCP tools accept a matching `condition` input. `cache()` now invalidates after a failed write too; `softDelete()` forwards a conditional delete of an already-trashed key; `dedup()` rejects every conditional mode.

A conditional mutation that commits before an awaited plugin rejects the call now surfaces as `FilesError.applied === true` (with `appliedEtag` for uploads) — on the rejected error, in `onError` / `onAction`, and in the `audit()` record — so callers can reconcile instead of retrying a predicate that can only conflict. A plugin that re-invokes `next()` after the native call failed gets that first failure as `cause`. `rejectConditional(op, plugin, reason)` is exported as the one veto shape for plugins with out-of-band side effects. Ordinary operations type `mode` as `undefined` (there never was an `"overwrite"` value to branch on).
26 changes: 23 additions & 3 deletions apps/web/docs/(concepts)/capabilities.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Capabilities
description: Query what an adapter can do up front with files.capabilities - branch on range reads, signed URLs, and server-side copy instead of catching a throw.
description: Query what an adapter can do up front with files.capabilities - including range reads, signed URLs, server-side copy, and native conditional operations.
---

The unified surface is the common subset every adapter implements, but adapters differ at the edges: some honor byte-range reads, some can mint a signed URL, some copy server-side. The wrapper already gates on these per-adapter — pass a `range` to an adapter that has no range primitive and it throws _before_ any provider call. `files.capabilities` turns that implicit knowledge into a queryable surface, so you can branch up front instead of discovering a limit by catching an error.
Expand Down Expand Up @@ -32,6 +32,19 @@ interface AdapterCapabilities {
multipart: boolean;
serverSideCopy: boolean;
signedUrl: { supported: boolean; maxExpiresIn?: number };
conditional: {
create: boolean;
replace: boolean;
exactRead: boolean;
delete: boolean;
copy: {
sourceEtag: boolean;
destinationCreate: boolean;
destinationReplace: boolean;
atomicSourceDestination: boolean;
};
multipart: { create: false; replace: false };
};
}
```

Expand All @@ -45,8 +58,15 @@ interface AdapterCapabilities {
| `multipart` | the adapter exposes a resumable / multipart upload primitive | `upload({ control })` |
| `serverSideCopy` | `copy()` runs server-side, with no body re-transfer through your process | `copy` |
| `signedUrl` | `url()` can mint a signed or tokenized URL — see below | `url` |
| `conditional` | the requested create, replace, exact-read, delete, or copy predicate has a provider-native primitive | `upload`, `download`, `delete`, `copy` |

Every field mirrors an operation the unified API actually has. There are deliberately no flags for `raw`-only territory (object versioning, checksums, conditional writes, POST policies) — advertising a flag for a non-operation would turn the matrix into a back-door spec, where a wrong flag is worse than no flag.
Every field mirrors an operation the unified API actually has. There are deliberately no flags for `raw`-only territory such as object versioning, checksums, or POST policies — advertising a flag for a non-operation would turn the matrix into a back-door spec, where a wrong flag is worse than no flag.

### `conditional`

Conditional capabilities are granular because providers often support only part of the surface. `create`, `replace`, `exactRead`, and `delete` map to their native single-object primitives. Conditional copy separately declares source ETag checks, destination create/replace checks, and whether the selected source plus destination predicates are evaluated atomically in one request. Invocation rechecks the declaration and fails before provider I/O when any required part is absent. These flags describe native adapter support; a composed plugin can still veto a conditional mode when its own policy cannot preserve that atomic boundary.

There is no probe-then-mutate fallback. See [Conditional operations](/docs/conditional-operations) for the API, ETag rules, provider matrix, and retry/observer semantics.

### `signedUrl`

Expand All @@ -60,6 +80,6 @@ signedUrl: { supported: boolean; maxExpiresIn?: number }

## How it's derived

`capabilities` is computed live from the adapter on every read, so a plugin that swaps behavior is always reflected. The first six fields read the exact per-adapter flags and optional methods the wrapper already gates on (`supportsRange`, `reportsUploadProgress`, `supportsDelimiter`, `supportsMetadata`, `supportsCacheControl`, and the presence of `resumableUpload`), so they can never drift from runtime behavior. `serverSideCopy` and `signedUrl` are declared by each adapter and default to the conservative value (`false`) when an adapter declares nothing — a caller that doesn't advertise reads as "no", never a wrong "yes".
`capabilities` is computed live from the adapter on every read, so a plugin that swaps behavior is always reflected. The first six fields read the exact per-adapter flags and optional methods the wrapper already gates on (`supportsRange`, `reportsUploadProgress`, `supportsDelimiter`, `supportsMetadata`, `supportsCacheControl`, and the presence of `resumableUpload`), so they can never drift from runtime behavior. Conditional support is derived from the adapter's optional native operation group and its explicit copy guarantees. `serverSideCopy`, `signedUrl`, and every absent conditional primitive default to the conservative value (`false`) — a caller that doesn't advertise reads as "no", never a wrong "yes".

If you're writing a custom adapter, set `supportsServerSideCopy` and `signedUrl` alongside the existing `supports*` flags to make your adapter introspectable; both are optional and advisory (they don't gate any operation).
109 changes: 109 additions & 0 deletions apps/web/docs/(concepts)/conditional-operations.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
title: Conditional operations
description: Use provider-native create, replace, exact-read, delete, and copy predicates without bypassing plugins, hooks, receipts, or client policy.
---

Conditional operations apply an ETag predicate in the provider's mutation or read request. They are the safe building blocks for create-only writes and compare-and-set updates: the SDK never implements them as `head()` / `exists()` followed by an unconditional operation.

```ts lineNumbers
// Create only when the key is absent.
const created = await files.upload("reports/q3.json", body, {
condition: { type: "create" },
});

// Replace only the generation returned by the create.
const replaced = await files.upload("reports/q3.json", nextBody, {
condition: { type: "replace", etag: created.etag },
});

// Read or delete exactly that replacement.
const exact = await files.download("reports/q3.json", {
condition: { etag: replaced.etag },
});
await files.delete("reports/q3.json", {
condition: { etag: exact.etag! },
});
```

A failed predicate rejects with [`FilesError`](/docs/api/errors) rather than weakening the request. An adapter without the requested native primitive also rejects before provider I/O.

## Conditional copy

A conditional copy checks the source generation and the destination predicate in one native provider request. Both predicates are required: the source must still have its supplied ETag, and the destination must either be absent or match its own supplied ETag.

```ts lineNumbers
// Publish the staged generation, but only if nothing is published yet.
const staging = await files.head("staging/report.json");
await files.copy("staging/report.json", "published/report.json", {
condition: {
source: { etag: staging.etag! },
destination: { type: "create" },
},
});

// Later: replace exactly the published generation with exactly the staged one.
const nextStaging = await files.head("staging/report.json");
const published = await files.head("published/report.json");
await files.copy("staging/report.json", "published/report.json", {
condition: {
source: { etag: nextStaging.etag! },
destination: { type: "replace", etag: published.etag! },
},
Comment thread
haydenbleasel marked this conversation as resolved.
});
```

There is no download-and-upload fallback for conditional copy. Check all of `sourceEtag`, the requested destination mode, and `atomicSourceDestination` under [`files.capabilities.conditional.copy`](/docs/capabilities) before planning one.

## ETag form

Pass the canonical bare strong ETag exposed by a conditional-capable adapter: for example `a1b2c3`, not `"a1b2c3"`. AWS S3 normalizes the values returned by `upload()`, `download()`, `head()`, and `list()` into this form. Empty values, weak validators (`W/…`), wildcards, quoted values, comma-separated lists, control characters, and excessively long values reject before provider I/O. Conditional uploads return a non-optional `etag` for the new generation; a provider commit whose response omits or corrupts that ETag rejects as an ambiguous outcome.

## Capabilities and current support

Each primitive is declared separately under `files.capabilities.conditional`. The declaration is conservative, and invocation checks it again.

These flags describe the adapter's native primitives. They are necessary, not sufficient, after composition: a plugin may still veto a mode when its own side effects cannot preserve the same atomic boundary.

```ts lineNumbers
const c = files.capabilities.conditional;

if (c.create && c.replace && c.exactRead && c.delete) {
// This adapter can support a native single-key CAS workflow.
}

if (
c.copy.sourceEtag &&
c.copy.destinationCreate &&
c.copy.atomicSourceDestination
) {
// Safe to perform a source-exact, destination-create copy.
}
```

The initial implementation supports canonical AWS S3 buckets through the AWS SDK adapter. A custom S3 `endpoint`, an `AWS_ENDPOINT_URL_S3` / `AWS_ENDPOINT_URL` redirect in the environment, or a shared-config `endpoint_url` does not inherit the claim, because S3-compatible services differ in which conditional headers they honor; the [`s3()` adapter's `conditional` option](/docs/adapters/s3) overrides that detection in either direction. The adapter also checks, per request, that the resolved hostname is AWS and that every predicate it set was serialized by the installed `@aws-sdk/client-s3`, and rejects the call otherwise. Cloudflare R2 is not supported.

The local filesystem adapter also reports every conditional capability as `false`. Its body and metadata sidecar are separate files, and a process-local lock plus a read-before-rename would not be compare-and-set against writers in another process. It fails closed until the storage layout can provide one native atomic commit boundary.

Conditional calls are single-key only. Bulk array calls, multipart uploads (`multipart: false`, the explicit opt-out, is fine), resumable `UploadControl`, signed uploads, and `move()` do not accept a condition. Use conditional copy followed by a separately conditioned delete only when your application can tolerate the two distinct commits; the SDK does not present that sequence as an atomic move.

## Plugins use the same operation families

Conditional calls traverse the existing ordered plugin onion under the same `upload`, `download`, `delete`, and `copy` kinds. Their `mode` distinguishes `create`, `replace`, `exact`, `match`, and `conditional` variants. That means existing per-verb encryption, compression, validation, metadata, audit, tracing, and usage handlers see the call instead of silently passing a new verb they do not recognize.

For a conditional root call, the SDK freezes the operation family, mode, and ETag predicates across every `next()` boundary. A plugin may transform the upload body, metadata, or returned value, or veto by throwing before `next()`. It cannot drop or change a predicate, reroute to another verb, call the native operation twice, or synthesize success without a native call. A rejected `next()` never reaches the provider, so a plugin that catches one and then calls `next(op)` with the original predicate receives the committed result as a success. Bundled plugins whose semantics require multiple backends or extra mutations reject the incompatible conditional modes before any of their provider I/O.

The built-in [`encryption()`](/docs/plugins/encryption) plugin is compatible with conditional create, replace, and exact read: conditional uploads store its transformed bytes, and exact reads traverse its inverse transform. As with every use of that plugin, its documented key and plaintext-compatibility policy still applies.

Other bundled plugins keep only the modes whose side effects preserve one native operation. Compression, content-type inference, and validation use their existing same-verb transforms. Exact reads bypass the ordinary cache, and the cache invalidates a key after any write settles — including a failed conditional one, since a `Conflict` is proof the cached ETag is stale. Soft delete rejects a conditional delete outside the trash prefix (a delete of an already-trashed key is a real delete and keeps its predicate); versioning rejects conditional writes and copies; deduplication rejects every conditional mode, because a pointer's ETag never reflects its content; and failover and tiering reject every conditional mode. Those vetoes happen before the plugin performs provider I/O.

## CLI and MCP

The [CLI](/docs/cli/commands#conditional-predicates) exposes the same predicates as flags — `--if-none-match` and `--if-match <etag>` on `upload`, `--if-match <etag>` on `download` and `delete`, and `--if-match <etag>` plus `--if-none-match` / `--dest-if-match <etag>` on `copy` — and the [MCP server](/docs/cli/mcp) accepts a `condition` input on `upload`, `download`, `delete`, and `copy` in the SDK's shapes. Both are single-key only and fail closed on adapters without native support, exactly like the SDK.

## Hooks, receipts, and terminal outcomes

`onAction` and `onError` keep their once-per-call semantics and `onRetry` its once-per-scheduled-retry cadence; all three include a redacted `condition` label, and no ETag predicate is copied into hook metadata (a successful conditional upload or exact read still exposes the committed ETag through `event.result`, as does an enabled receipt). A successful conditional mutation receives the ordinary upload, delete, or copy receipt with the same condition label. Reads and failed calls have no receipt.

Hooks and receipt delivery are fire-and-forget: a hook that throws cannot change the result after the provider commits. An awaited plugin is different. If it calls `next()`, observes the committed result, and then throws, the public call rejects, `onError` and an error `onAction` fire, and no success receipt is emitted. The provider mutation cannot be rolled back, so that outcome is applied-but-unacknowledged: the rejected [`FilesError`](/docs/api/errors) carries `applied: true` (and `appliedEtag` for uploads), the same flag reaches `onError` / `onAction` and the [`audit()`](/docs/plugins/audit) record, and the right recovery is an exact read rather than a retry of the same predicate — which can now only conflict.

Retries have the same ambiguity at the network boundary. A response can be lost after a provider commit; a retry of the same predicate can then fail because the first request already changed the object. Conditional retries never turn into an unconditional request, but callers that receive an error must still reconcile before assuming nothing changed.
4 changes: 3 additions & 1 deletion apps/web/docs/(concepts)/receipts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Receipts ride on the existing [`onAction`](/docs/api/onaction) hook as an additi
- the call is a mutating verb (`upload`, `delete`, `copy`, `move`), and
- the call **succeeded**.

Reads, `signedUploadUrl`, failures, bulk array calls (which aggregate many objects into one event), and every instance with receipts off leave `event.receipt` unset - so an existing `onAction` consumer that never opted in sees the exact payload it always has.
Reads, `signedUploadUrl`, failures, bulk array calls (which aggregate many objects into one event), and every instance with receipts off leave `event.receipt` unset - so an existing `onAction` consumer that never opted in sees the exact payload it always has. A successful [conditional mutation](/docs/conditional-operations) uses the ordinary `upload`, `delete`, or `copy` receipt and adds its redacted `condition` label; predicate ETags are never copied into the receipt.

Every field except `sha256` is **derived** from the work the SDK already does for the hook - the timing, the adapter name, the caller-facing key, and `bytes` / `etag` read straight off the [`UploadResult`](/docs/api/upload). Turning receipts on with `receipts: true` therefore adds no per-call cost.

Expand All @@ -60,6 +60,8 @@ const files = new Files({

With `receipts: true` (or `{ sha256: false }`), the body is never read and no hash is taken.

Receipt delivery remains observational. If `onAction` throws, the committed operation still succeeds. If an awaited plugin throws after the native operation has committed, the public call rejects and no success receipt is emitted; reconcile that applied-but-unacknowledged outcome with an exact read.

### Plugins that transform the body

The fingerprint is taken **before** any plugin runs. If you compose the instance with a body-transforming plugin - [`encryption`](/docs/plugins/encryption) writes ciphertext, [`compression`](/docs/plugins/compression) writes compressed bytes - the bytes on disk differ from `sha256`. That's deliberate: it's the hash of the content you handed in, and it matches what a [`download`](/docs/api/download) gives back, since reads reverse the same transforms. So it's the value a round-trip check can verify - and the only stable one, since `encryption` uses a fresh key per object and would otherwise hash differently on every upload of identical content.
Expand Down
Loading