Skip to content

Split S3/R2 conditional capabilities and add conformance tests - #21

Merged
kauandotnet merged 5 commits into
mainfrom
codex/issue-5
Aug 14, 2026
Merged

Split S3/R2 conditional capabilities and add conformance tests#21
kauandotnet merged 5 commits into
mainfrom
codex/issue-5

Conversation

@kauandotnet

@kauandotnet kauandotnet commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • split coarse S3 mutation/copy claims into exact create, replace, delete, read, source-copy, destination-copy, atomic-promotion, multipart-completion, physical-key, and signed-upload-policy capabilities
  • declare AWS S3 and Cloudflare R2 independently; force unknown, custom, and noncanonical S3 endpoints fully read-only until a branded explicit conformance profile is supplied
  • publish a runner-neutral provider conformance harness covering exact capabilities, fail-closed unsupported paths, canonical ETags, replayable provider cursors, complete copy-predicate matrices, key budgets, signed-upload policy, and log-safe errors
  • bind S3 trust to package-private client and exact adapter-surface provenance, and enforce complete physical keys at the innermost adapter dispatch boundary
  • enforce signed-upload constraints at signing time: MIME-bound PUT, bounded AWS POST, fail-closed unsupported R2/custom constraints, and exact-key protection from AWS ${filename} policy expansion

Supports nestm-dev/artifact-studio#5

Required coordinated consumer PR: pending. Artifact Studio will be rebased and tested against the merged Storage API, then linked here; this PR intentionally does not close the Artifact Studio issue.

Provider matrix

Capability AWS S3 general-purpose Cloudflare R2 stable Unknown/custom default
Conditional create / replace yes / yes yes / yes read-only
Conditional delete ETag absent read-only
Conditional read ETag + version ETag only read-only
Source-conditioned copy ETag + version ETag only read-only
Destination create / replace yes / yes absent read-only
Atomic combined promotion provider evidence + complete harness matrix absent read-only
Conditional multipart completion create + replace absent read-only
Complete physical-key budget 1,024 bytes 1,024 bytes explicit verified profile
Signed-upload policy MIME-bound PUT; MIME + byte-range POST MIME-bound PUT; no POST size range false / false unless verified

Custom endpoints may enable only operations and signed-upload constraints proven by their conformance evidence. Undeclared operations fail before provider dispatch. withS3Capabilities() is single-use per raw SDK client, and generic composition rejects undecorated structural S3 adapters.

Security and correctness

  • Application-facing ETags are canonical bare strong tokens. Weak, wildcard, list-shaped, quoted, control-bearing, non-ASCII, and oversized forms fail before dispatch; each S3 request field receives exactly one wire quote pair.
  • Ordinary and conditional provider failures expose stable codes/messages and retry flags without raw bodies, request IDs, SDK metadata, causes, or physical tenant prefixes.
  • Native AWS clients ignore environment/shared-config endpoint redirects. Explicit endpoint and CDN provenance follows the actual package-created adapter, including configJson.
  • Native explicit profiles may narrow but cannot widen the immutable AWS operation, policy, or 1,024-byte key ceiling, regardless of adapter display name.
  • Profile verification and adapter provenance are package-private and identity-bound; reflected symbols, aliases, replaced methods/raw clients, and broader capability surfaces cannot forge or widen them.
  • Signed PUTs that request contentType include content-type in X-Amz-SignedHeaders. Bounded AWS uploads use exact content-length-range and MIME POST conditions. S3 lower-only minSize, R2 size bounds, and undeclared custom constraints fail before credentials resolve or a signer runs.
  • Bounded POST physical keys ending in literal ${filename} fail before signing, preventing AWS SDK template expansion from widening an exact key into a prefix policy.
  • The final internal Files plugin validates transformed object keys, copy/move pairs, list prefixes, and files-sdk-derived search walks after supported user plugins run.
  • List cursors are opaque, non-consuming, and replayable for identical options against unchanged provider state. Every advertised source (etag, version) × destination (create, replace) promotion pair is exercised; provider evidence remains required for a private one-linearization-point claim.

No credentials, database migration, RLS change, tenant-model change, or authorization-policy change is included. Tenant isolation is unchanged; key, ETag, endpoint, error, and signing boundaries are stricter.

API and deployment migration

This includes a minor Changeset.

  • Replace legacy conditionalMutation / conditionalCopy checks with exact capability fields.
  • Pass source ETags to workspace copy/move calls and refresh persisted quoted ETags from provider metadata.
  • Route raw S3 composition through package s3() + withS3Capabilities() or createS3StorageDriver().
  • Supply an explicit branded provider profile for custom/noncanonical S3 mutations. Omitted signed-upload policy becomes {contentType:false,sizeRange:false}.
  • Treat R2 as PUT-only for signed uploads; it does not satisfy the gateway's bounded POST requirement.
  • Require @aws-sdk/client-s3 >= 3.919.0, the first release that serializes destination CopyObject conditions. Other AWS signing peers retain their declared >=3.700.0 floors, verified independently by the packed consumer.

Verification

Final local evidence on 0ac8922:

  • focused signer suite: 38 unit tests passed
  • focused gateway suite: 28 tests passed, 1 intentional skip
  • pnpm run test: 216 unit tests passed; 55 E2E tests passed, 4 live-provider tests skipped explicitly
  • pnpm run check: oxlint, full Prettier check, and tsc --noEmit passed
  • pnpm run build: passed
  • pnpm exec publint --strict: passed
  • pnpm run test:packed: passed with exact peer minima (client-s3@3.919.0; lib-storage, s3-presigned-post, and s3-request-presigner at 3.700.0), real serialized copy headers, MIME-bound PUT signatures, decoded bounded POST conditions, and fail-closed R2/template paths
  • git diff --check: passed

AWS S3, Cloudflare R2, and custom live-provider suites remain explicit opt-in skips without disposable credentials. Filesystem conformance and deterministic versioned/fail-closed fixtures run in the default suite; live claims remain disclosed pending credentialed execution.

@kauandotnet
kauandotnet marked this pull request as ready for review August 14, 2026 20:43
@kauandotnet
kauandotnet merged commit d996b92 into main Aug 14, 2026
6 checks passed
@kauandotnet
kauandotnet deleted the codex/issue-5 branch August 14, 2026 20:43
@kauandotnet

Copy link
Copy Markdown
Contributor Author

Artifact Studio consumer draft: https://github.com/nestm-dev/artifact-studio/pull/20

This draft consumes the merged Storage #21 API and closes artifact-studio issue #5 when merged. Storage PR #20 is the separate issue-#4 cursor track and is not a dependency of this consumer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant