Repository navigation
Release - #227
Release#227
Conversation
* feat: implement tiered mode PUT/GET/HEAD/DELETE served entirely from cache with authoritative misses (ETag over decoded bytes, aws-chunked unwrap, tombstone-aware stores, budget-reserved admission), minus block-aligned storage, listings and bucket ceremony. Groundwork for tiered mode (#201); full origin-less mode stays parked on PR #191. * feat: add TAG_MODE operating-mode enum (transparent|signing|tiered) mode supersedes upstream.transparent_proxy (kept as a deprecated alias; contradictions are startup-fatal). Tiered mode requires the cache, is whole-object only — explicit block_caching_enabled=true is rejected and the default flips to off. * feat: tiered store mode routing (#201) Local metadata is authoritative for existence in both tiers: a meta miss answers NoSuchKey with zero upstream traffic. Small PUTs (≤ cache.size_threshold) store whole objects via the local-store engine; large PUTs pass through and stamp a BodyUpstream meta marker that serves HEADs locally, leaving GET body forwards as the mode's only body traffic. Cross-tier overwrites clean up the displaced version (async upstream DELETE / local invalidation). Unvalidated requests forward unchanged, as in transparent mode.
A `mode` input (transparent default | tiered) selects TAG_MODE. Tiered is whole-object only, so block caching is forced off regardless of cache_mode (tiered + block caching is a fatal config), and size_threshold defaults to 16 MiB — the benchmark's objects are kept at or below that (the GET RANGE object defaults to 16 MiB in tiered instead of 256 MiB) so they are stored and served entirely by the local tier, which is the tiered path worth measuring. TAG_MODE/TAG_CACHE_SIZE_THRESHOLD reach the binary through the environment it inherits, like the existing max_populate_memory knob, so no Makefile change is needed. Explicit overrides still win.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d8bf65d. Configure here.
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…rge retries once (#228) * fix: tiered CopyObject converges under its pre-forward version; converge retries once CopyObject was the last mode-blind mutating handler whose post-success converge out-deleted racing writers in tiered mode: a small PUT acked while the copy was in flight — the local tier's only copy — lost its metadata behind the copy's 200. It now claims the key and captures a pre-forward token like the forwarded DELETE, converging only the state the copy displaced. The tiered converge also retries once on a transient error (the discipline the proxy-mode converge already had), keeping the version guard: a converge that never lands leaves deleted metadata authoritative until TTL. Dead unreachable branch removed from the engine's PUT error path. * test: tiered converge retry — same version token, exactly two attempts
|
🎉 This PR is included in version 1.22.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
3 participants
Cuts v1.22.0: tiered store mode (#203), its benchmark support (#226), and the release-review fixes (#228: tiered CopyObject converges under its pre-forward version; the tiered converge retries once). The full CAS coordination line shipped in v1.21.0 and is running in production; this release adds the mode built on it.
Tiered store mode (
mode: tiered/TAG_MODE=tiered) — TAG as a two-tier cache in front of a cheap capacity-priced upstream bucket. Local metadata is authoritative for existence (a miss is a local NoSuchKey, the caller's cue to fall back to its system of record). Objects at or belowcache.size_thresholdare the local tier: PUT streams the body into the embedded cache (never buffered, fixed memory regardless of size), GET/HEAD/DELETE are served entirely locally — zero upstream traffic. Larger objects are the upstream tier: the PUT passes through and a metadata marker is stamped, HEAD answers from the marker, GET forwards the body. Multipart completions stamp a marker (immediately, upgraded with a background HEAD); re-tier-on-read heals mis-tiered small objects; cross-tier overwrites clean up the displaced upstream copy with a conditional DELETE.Startup contract: tiered forces CAS coordination (auto-selected; an explicit
legacy_coordination: trueis fatal) and whole-object caching (block_caching_enabled: trueis fatal). Off by default — existing deployments are unaffected; no config change needed to upgrade.Hardening: four adversarial review passes plus the bot rounds closed every finding — the acked-write-loss family (all converge/sweep/cleanup paths version-guarded), the streaming PUT's abort path (panic-safe deferred reclaim, ambiguous-commit rule), RFC 7232/9110 conditional semantics (ETag lists, strong If-Match, 412s only where authoritative), Content-MD5 validation, request ids on TAG-originated errors.
Observability:
tag_requests_total/tag_request_duration_secondsgainmodeandsource(local|upstream) labels;tag_tiered_cleanup_total/tag_tiered_retier_totalcounters. Dashboards usingsum byare unaffected.Validation: dedicated
s3-compat-tests (tiered)CI job (79-test tiered profile, green); warp A/B at 4 MiB objects, two concurrent pairs — HEAD ~5× faster (authoritative local metadata), PUT ~1.14× with a consistently tighter p99, GET throughput a wash with steadier latency, GET-RANGE ~0.72× (block caching's home turf, not tiered's target).Docs:
docs/tiered-mode.md(semantics, credential requirement — TAG's credentials need delete permission on the cache bucket — limitations: listings/copies/object sub-resources pass through and cannot see local-tier objects).🤖 Generated with Claude Code
https://claude.ai/code/session_01L5DLvDTo9dhD38YQXhHpU9