fix: make workspace cursors replica-safe

.changeset/fix-workspace-cursor-continuity.md

@@ -0,0 +1,20 @@
+---
+'@nestm/storage': minor
+---
+
+Add injectable, replica-safe workspace pagination cursors. The workspace now
+binds versioned cursor payloads to stable store, mount, tenant/workspace, prefix,
+operation, query, limit, and expiry context; authorizes non-consuming replay
+before that expiry; and rejects altered or cross-context continuations.
+Successful continuation still depends on the embedded provider cursor remaining
+valid and available.
+
+Export an AES-256-GCM key-ring codec for stateless multi-replica deployments and
+an asynchronous byte-payload codec contract for shared durable opaque-token
+stores. Cursor payloads and tokens are bounded, provider continuations remain
+opaque, and pagination fails closed when no cursor mechanism is configured.
+Compatible replicas rely on the universal driver contract for non-consuming,
+instance-portable provider cursors whose position is independent of page size
+while the provider token remains valid. Cursor expiry is not a provider-token
+retention, snapshot-isolation, or uptime promise; provider invalidation remains
+an operational list failure.

README.md

@@ -123,12 +123,46 @@ control characters, repeated separators, and `.` or `..` segments are rejected
 rather than normalized. Keys and provider cursors returned by a driver are also
 checked before they are converted back to logical paths.
 
+Pagination requires a server-owned cursor configuration. The built-in
+`Aes256GcmStorageWorkspaceCursorCodec` produces versioned, authenticated,
+encrypted tokens that can resume on another request, process, or replica when
+every replica constructs an equivalent codec from the same key ring and uses
+the same stable store identity, physical prefix, mount ID, trusted scope, and
+effective limits. Use one codec instance per process, use a dedicated 32-byte
+key, retain rotated decryption keys for at least one cursor TTL, and derive
+`mountId` and `scope` only from authenticated server context.
+
+The underlying driver must also implement the universal replayable list-cursor
+contract against the same logical backend namespace: its cursor cannot be
+consumed or tied to one driver instance. While the provider cursor remains
+valid and available, an outer cursor can be retried until its authenticated
+expiry. That expiry is only an authorization ceiling: it does not extend a
+provider token's lifetime or promise snapshot isolation, provider availability,
+network access, or valid credentials. Provider invalidation is an operational
+list failure, and concurrent object changes remain subject to provider
+continuation semantics. Without a codec, single-page operations still work but
+a continuation fails closed.
+
 ```ts
-import { mountStorageWorkspace } from '@nestm/storage/workspace';
+import {
+  Aes256GcmStorageWorkspaceCursorCodec,
+  mountStorageWorkspace,
+} from '@nestm/storage/workspace';
+
+// cursorKey is a separately validated 32-byte secret from deployment config.
+const cursorCodec = new Aes256GcmStorageWorkspaceCursorCodec({
+  activeKeyId: 'v1',
+  keys: { v1: cursorKey },
+});
 
 const workspace = mountStorageWorkspace(agentFiles, {
   // Use an opaque server-derived run id, never a value selected by the model.
   prefix: `workspaces/${runId}`,
+  cursor: {
+    codec: cursorCodec,
+    mountId: `agent-workspace:${runId}`,
+    scope: `organization:${organizationId}/workspace:${workspaceId}`,
+  },
   permissions: [
     'list',
     'read',
@@ -140,6 +174,8 @@ const workspace = mountStorageWorkspace(agentFiles, {
     'delete',
   ],
   limits: {
+    cursorTtlMs: 5 * 60 * 1000,
+    maxCursorBytes: 4096,
     maxReadBytes: 1024 * 1024,
     maxWriteBytes: 1024 * 1024,
     maxPageSize: 100,
@@ -233,6 +269,8 @@ import type { ToolSet } from 'ai';
                 // A validated, opaque coordinate from trusted auth/run state.
                 // It is never accepted from a prompt or tool input.
                 prefix: context.storagePrefix,
+                // Includes the singleton codec plus stable mountId and scope.
+                cursor: context.cursorConfiguration,
                 permissions: [
                   'list',
                   'read',
@@ -514,10 +552,22 @@ receive no inferred conditional capabilities.
 - pause/resume/abort through `StorageUploadControl`.
 
 Provider list cursors are opaque, non-consuming continuation tokens. Replaying
-the same cursor with the same list options against unchanged provider state
-must return an equivalent page and continuation cursor. This contract lets a
-caller safely retry or replay pagination; it does not promise snapshot
-isolation across concurrent provider mutations.
+the same cursor and page limit against unchanged provider-visible state must
+return an equivalent page and continuation position, even after a descendant
+cursor has been used. A cursor is bound to the logical store, `prefix`, and
+`delimiter`, but not to `limit`, retries, timeout, or abort signal; callers may
+change those transport/page-size options while resuming the same position.
+
+The cursor must work through a newly constructed compatible driver targeting
+the same backend namespace while the provider token remains valid and
+available; it cannot depend on process-, client-, or session-local state. An
+adapter for a consuming or instance-bound provider token must materialize a
+stable continuation before it can provide conforming paginated
+`StorageDriver.list` results. This contract lets a caller safely retry, replay,
+or resume pagination on another replica. It does not promise a provider-token
+lifetime, snapshot isolation across concurrent mutations, or provider,
+network, credential, or authorization availability. Provider invalidation is
+an ordinary list-operation failure.
 
 Downloads are streaming by default:
 

SECURITY.md

@@ -56,11 +56,40 @@ untrusted agent code. A tenant id, run id, prefix, provider cursor, snapshot id,
 or fork id supplied by a model is not a safe mount coordinate.
 
 The workspace accepts only canonical mount-relative POSIX paths and rechecks
-every key returned by a driver before unscoping it. Its cursors are opaque and
-bound to the mount and query. Permissions, byte limits, result limits, and
-conditional mutation preconditions are enforced inside the capability; tool
-omission and user approval are additional workflow controls, not the
-authorization boundary.
+every key returned by a driver before unscoping it. Pagination cursors are bound
+to the store, normalized physical prefix, stable mount identity, trusted
+tenant/workspace scope, operation, complete effective limits, normalized query,
+and expiry. Provider continuations and prefixes stay inside the encrypted or
+server-side payload. Both layers are non-consuming: a durable opaque token
+store must read rather than consume a record, and an embedded provider cursor
+must remain replayable through a fresh compatible driver against the same
+backend namespace while that provider cursor remains valid and available.
+Provider cursors cannot depend on process-local state; an adapter for a
+consuming or instance-bound backend token must materialize a stable continuation
+before exposing paginated `StorageDriver.list` results.
+
+Under unchanged provider-visible state, cursors are reusable while their
+provider continuation remains valid and available. The authenticated expiry is
+an authorization ceiling, not a guarantee of provider-token lifetime, snapshot
+isolation, provider/network availability, or valid credentials. Provider
+invalidation remains an operational failure, and concurrent mutations remain
+subject to provider ordering, duplicate, and omission semantics.
+
+Production pagination must configure either the built-in AES-256-GCM codec with
+a dedicated shared 32-byte key ring or an authenticated shared durable token
+store. Do not reuse an authentication, session, storage-provider, or encryption
+key from another purpose. All replicas must use the same stable identities and
+key ring, compatible driver configuration, and logical backend namespace. New
+cursors use the active key id; retain prior keys for at least the maximum cursor
+TTL during rotation. Dropping a key, changing a binding field, or restarting
+with a different ephemeral key intentionally invalidates outstanding cursors.
+Tokens and decoded payloads are bounded, and malformed, altered, expired,
+cross-query, cross-operation, cross-workspace, and cross-store cursors fail
+closed.
+
+Permissions, byte limits, result limits, and conditional mutation preconditions
+are enforced inside the capability; tool omission and user approval are
+additional workflow controls, not the authorization boundary.
 
 Conditional mutations can still have an ambiguous outcome when a remote
 provider commits and then loses or violates its response, or when a configured

docs/provider-conformance.md

@@ -4,8 +4,8 @@
 `createStorageProviderConformanceCases()`, a runner-agnostic contract harness
 for the conditional storage boundary. It verifies the exact declared
 capability matrix, supported operations, fail-closed unsupported operations,
-the complete physical-key byte budget, conflict normalization, and public
-error sanitation.
+replayable and replica-portable list cursors, the complete physical-key byte
+budget, conflict normalization, and public error sanitation.
 
 Every failure observed by the harness must be log-safe as a nested object, via
 `util.inspect`, and via JSON serialization. Provider bodies, request/host IDs,
@@ -50,16 +50,41 @@ discarded and refreshed from provider metadata before mutation tests run.
 
 ## List cursor replay contract
 
-Provider list cursors are opaque and non-consuming. Reusing the same cursor
-with the same prefix, delimiter, and page limit against unchanged provider
-state must return an equivalent page and the same continuation state. The
-conformance harness creates three objects, advances beyond a cursor-backed
-page, then replays the original cursor and compares the full page metadata and
-next cursor. A consuming or unstable provider cursor fails conformance; callers
-must not advertise replayable higher-level cursors for that provider.
-
-This is an idempotent-replay guarantee, not snapshot isolation. Results may
-change when another actor mutates the provider namespace between list calls.
+Provider list cursors are opaque, non-consuming, and portable. They are bound
+to the original `prefix` and `delimiter`, but not to `limit`: callers may choose
+a different supported page size when they resume. Against unchanged provider
+state, reusing a cursor must resume from the same logical position even after a
+caller has followed a descendant cursor.
+
+The same token must work through an independently constructed compatible
+`StorageDriver` and `StorageClient` that address the same logical store with the
+same backend configuration while the provider cursor remains valid and
+available. A driver therefore cannot keep cursor state only in process memory
+or bind it to one client/session.
+
+Equivalent continuation positions do not require byte-identical continuation
+tokens. The harness always runs a delimiter-free case and additionally covers
+common prefixes when the driver advertises delimiter support. It follows a
+one-entry page and its descendant, replays the ancestor, changes the page limit,
+and repeats both page sizes through a fresh replica. It compares page metadata,
+common prefixes, and the page reached by each returned continuation instead of
+comparing opaque token bytes. A partial page with matching entries remaining
+must include a cursor; silently omitting one is a contract failure, not an
+opt-out. Fixtures must provide `createReplica()` using a new driver and client
+for the same root, bucket, logical store name, and provider configuration; the
+harness shuts that client down even when a case fails.
+
+A one-shot, session-bound, or otherwise consuming custom driver violates the
+public `StorageDriver.list` contract and fails this case. Such a driver must be
+adapted to issue stable portable cursors before it can implement
+`StorageDriver`.
+
+This is an idempotent-replay guarantee, not snapshot isolation or a cursor
+lifetime/uptime SLA. Results may change when another actor mutates the provider
+namespace between list calls. Provider token invalidation and ordinary backend,
+network, credential, or authorization failures may still make a list request
+fail. The deterministic harness proves immediate replay and replica portability;
+it does not time-prove provider cursor retention.
 
 ## Combined copy atomicity
 
@@ -220,7 +245,8 @@ register each returned case with their test runner. A case may return
 `{ status: "skipped", reason }` for a provider prerequisite such as bucket
 versioning; translate that result into the runner's native skip mechanism.
 
-Fixtures can supply provider-aware cleanup and a version resolver:
+Fixtures supply an independently constructed replica for cursor portability
+checks and may also supply provider-aware cleanup and a version resolver:
 
 ```ts
 import {
@@ -233,6 +259,7 @@ const options: StorageProviderConformanceOptions = {
   expected: verifiedCapabilities,
   createFixture: async () => ({
     client: await createDedicatedTestClient(),
+    createReplica: createAnotherClientForTheSameStore,
     cleanup: deleteEveryTestObjectIdentity,
     resolveVersion: resolveCurrentProviderVersion,
   }),