Skip to content

storage: tenant document lifecycle tiers and S3 actor backend - #201

Draft
lost-rob0t wants to merge 35 commits into
masterfrom
feat/tenant-document-lifecycle-storage
Draft

lost-rob0t wants to merge 35 commits into
masterfrom
feat/tenant-document-lifecycle-storage

Conversation

@lost-rob0t

Copy link
Copy Markdown
Owner

Summary

Adds tenant-aware document lifecycle APIs and a Star-Lang/Sento document-storage actor.

Lifecycle API

  • GET /api/v1/documents/:id/lifecycle (documents:read)
  • PUT /api/v1/documents/:id/lifecycle (documents:write)
  • logical tiers: hot, warm, cold, archive
  • callers choose the logical tier only; backend/bucket/endpoint/credentials stay server-owned

Storage model

  • CouchDB remains the tenant/dataset authorization, metadata, revision, and index plane
  • hot: full CouchDB document
  • warm: full CouchDB document + mirrored object payload
  • cold / archive: CouchDB authorization/index stub + authoritative external payload
  • direct GET/update hydrates only after the stored CouchDB stub is authorized
  • ordinary document/search egress redacts extensions._server_storage; lifecycle API exposes placement explicitly

Backends / actor runtime

  • pluggable star.storage protocol (put/get/delete/head)
  • in-memory backend for hermetic tests
  • S3-compatible backend with AWS Signature V4 using existing ironclad, dexador, and quri dependencies
  • path-style S3 addressing supports AWS S3 and compatible services such as MinIO
  • pinned Sento document-storage actor
  • Star-Lang actor contract in spec/document-storage.star

Ingest behavior

  • new documents adopt server-configured tenant default tiers
  • updates preserve an existing explicit tier, including explicit hot
  • Rabbit delivery is not acknowledged until placement completes
  • external transition writes object first, CouchDB pointer/stub second, then cleans old object copy

Configuration

  • STAR_STORAGE_DEFAULT_TIER
  • STAR_STORAGE_TENANT_TIERS
  • STAR_STORAGE_TIER_BACKENDS
  • STAR_S3_REGION
  • STAR_S3_ENDPOINT
  • STAR_S3_BUCKET
  • STAR_S3_ACCESS_KEY_ID
  • STAR_S3_SECRET_ACCESS_KEY[_FILE]
  • STAR_S3_SESSION_TOKEN[_FILE]

Includes capabilities/OpenAPI/client-contract integration, docs, and a required hermetic lifecycle/storage test suite.

Copy link
Copy Markdown
Owner Author

Runtime/storage reconciliation — this branch is broader than the now-authoritative #206 first slice and should not be merged as-is.

#206 explicitly owns the data-tier/lifecycle boundary and narrows the first production slice to a pure deterministic library: backend protocol + in-memory fixture, tier registry, storage metadata, ordered lifecycle evaluator, verified fake-backend move, trusted init example, and docs. It explicitly excludes REST APIs, background migration, provider provisioning, production migration, and defers the S3 adapter until a separately justified signer/transport slice.

PR #201 exact head cd8ad310cdc102ddfbb40ea08fe34a5f6d7d32c2 currently bundles /api/v1 lifecycle APIs, S3 SigV4 transport, a Sento storage actor/Star-Lang contract, ingest placement semantics, and authoritative external-payload stubs. Those are valuable experiments/fixtures, but they cross multiple #206 follow-on boundaries and also deepen /api/v1 drift.

Gate state: schema lock, Operational Salvage, and Container Stack are green; Smoke Tests and Docstring Coverage are red. The #206 preflight comment also records that the required five independent issue voters were not executed, so AGENTS.md still blocks production mutation/merge.

Salvage when the gate opens: the backend protocol, in-memory backend tests, checksum-before-delete move invariants, lifecycle fixtures, and any provider-neutral metadata model that fits #206. Keep S3/actor/API integration out of the first recovered slice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant