Block cache, coalescing, and single-flight

CHANGELOG.md

@@ -13,9 +13,102 @@ one.
 
 ## Unreleased
 
-Nothing yet. `v0.3.0`'s scope is the block cache, and its definition of success
-is the table in [BASELINE.md](docs/reference/BASELINE.md) § *What the next
-release has to move*.
+`v0.3.0`'s scope, the block cache, is implemented and unreleased. Its definition
+of success was the table in [BASELINE.md](docs/reference/BASELINE.md) *What the
+next release has to move*, and all four rows moved the way that table asked:
+the clustered header-and-index read went from 18 requests to 3, the bounded
+query's `amplification` went above 1.0 with `bytesOverFetched` to account for
+it, eight parallel readers of one asset went from 152 requests to 25, and the
+full sequential read did not move at all.
+
+### Added
+
+- **`libs/usd-asset-cache`**, the block cache, as a decorator over
+  `AssetReader`. It links `usdAssetIo` and nothing else: no transport, no
+  backend, no OpenUSD. Reads are expanded to whole blocks and served from them;
+  the final block of an asset is stored at its true length and never padded;
+  adjacent and near-adjacent fetches are merged into one request; concurrent
+  readers that miss the same block issue one request and the rest wait; blocks
+  are evicted LRU under a process-wide budget shared across assets; and a read
+  large enough to be a streaming pass bypasses the whole thing.
+  [CACHE.md](docs/architecture/CACHE.md) is the contract and the module
+  [README](libs/usd-asset-cache/README.md) states what it refuses to own.
+- **A validator-keyed `CacheKey` from the first commit.** The key is
+  `resolvedIdentifier + validator + blockSize + blockIndex`, and the rule it
+  exists for is that equal identifiers never imply equal content: two revisions
+  published at one URL are two cache identities. The validator is an opaque byte
+  string here — never parsed, never compared to an `ETag`, never read for
+  recency — and exactly one other field of it is read, `strength`, exactly once.
+- **Single-flight, across readers and not only across threads.** Eight threads
+  missing one block issue one request; so do eight independent readers of one
+  revision, because they share an identity and therefore share the store. That
+  second case is the one that moved the parallel-readers baseline, and it is why
+  the store is process-wide rather than per reader.
+- **The cache counters of [METRICS.md](docs/architecture/METRICS.md) §2.2**,
+  populated: hits, misses, partial hits, requests saved by coalescing and by
+  single-flight, `bytesOverFetched`, evictions, and a resident high-water mark.
+- **A third row in the shared boundary suite**, `cache over local`. The cache is
+  not a transport, so it is not a fourth backend — it is the same local backend
+  with a decorator on top, and entering it there is what makes "byte-for-byte
+  equivalence with the uncached path over the full suite" an assertion rather
+  than a claim. Every case runs unchanged.
+- **`tests/cache-tuning`**, the measurement that chose the constants: a sweep of
+  five block sizes against four coalescing gaps over four access patterns,
+  against a real socket, with every byte verified. Recorded in
+  [BLOCK_POLICY.md](docs/reference/BLOCK_POLICY.md), which also labels the two
+  constants that were *not* measured as the bounds they are.
+- **The four cache variables of
+  [CONFIGURATION.md](docs/reference/CONFIGURATION.md) §2**, read once when the
+  resolver is constructed. A block size that is not a power of two is rounded
+  down and the rounding is reported; a value outside the bounds is refused
+  rather than clamped.
+- **`ReaderMetrics::AbsorbTransport` and `DetachFromRegistry`**, so that a
+  decorated stack reports one counter set instead of two.
+- **Sanitizer coverage over all of it**, which needs no new lane: the module
+  tests, the boundary row, and the tuning sweep are `libs/` and `tests/`, which
+  is what `core-asan` and `core-tsan` already cover. Both are green over the
+  whole core tree, 25 of 25 each, under GCC 15.2.
+
+### Changed
+
+- **The resolver decorates every asset it opens.** `plugins/http-resolver` links
+  `usdasset::cache`, which [WORKSPACE.md](docs/architecture/WORKSPACE.md) §2 has
+  admitted since the workspace contract was written and this release is the
+  first to take. `httpResolver_stage` now asserts from the fixture server's log
+  that a 4 KiB window out of a megabyte costs a block and not the megabyte —
+  a bound rather than an exact range, because the exact-bytes property is what
+  CACHE.md §3 trades away on purpose.
+- **The recorded I/O baseline holds every scenario twice**, with the cache and
+  without it, in one run of one harness. METRICS.md §6 asks a release that
+  changes I/O behavior for the counter values before *and* after, and this is
+  the first release that changes them on purpose. `tests/baseline` gained the
+  cache on its link line and a set of assertions for the cached rows: the
+  server's log is the independent witness for the byte count, the request count
+  is asserted to be below the uncached row, and the full sequential read is
+  asserted to be *identical* rather than merely close.
+- **The boundary suite's mid-read revision case now asserts bytes as well as a
+  status**, and asserts them in the right place. It reports `AssetChanged` for a
+  read at an offset the reader has not read before; for a range it has already
+  read, it accepts either `AssetChanged` or byte-for-byte what the first read
+  returned, and rejects the new revision's bytes. This is a strengthening rather
+  than a relaxation — the byte comparison is new, and the old case would have
+  passed a backend that rebound *and* reported `AssetChanged`. It is also what a
+  reader with a cache under it can satisfy honestly: §2.1 of
+  [ASSET_READER.md](docs/architecture/ASSET_READER.md) says a reader that
+  *observes* a changed validator fails subsequent reads, and a hit observes
+  nothing and returns the revision the reader is bound to.
+- **`selectivity` on the bounded query went from 0.0025 to 0.0112**, on purpose.
+  Alignment converts request count into transferred bytes, and one percent of a
+  128 MiB asset to answer a query against it is still the sentence the
+  architecture is made of. The cost is in `bytesOverFetched`, which is reported
+  beside the saving rather than instead of it.
+- The metrics dump prints the cache block and the two cache ratios, including
+  the zeroes. A dump that hid them would make "no cache ran" and "the cache
+  never hit" the same output.
+
+### Fixed
+
+Nothing. No defect in `v0.2.0` was found by this work.
 
 ## `v0.2.0` — 2026-08-20
 

README.md

@@ -31,8 +31,11 @@ index, and the chunks actually in view — not 10 GB.
 
 **`v0.2.0` is released: a `UsdStage` opens over HTTP. The read contract, the
 local backend, the shared boundary suite, the hostile-server corpus, the HTTP
-backend, and the `ArResolver` bundle are in the tree and passing. There is no
-cache.**
+backend, and the `ArResolver` bundle are in the tree and passing.**
+
+**`v0.3.0` is in the tree and unreleased: the block cache. A clustered read of a
+remote asset now costs three requests where it cost eighteen, and eight parallel
+readers of one asset move what one reader moves.**
 
 That ordering is the point. `v0.1.0` shipped a local file reader, which is not
 interesting; what was interesting is that it arrived with the harness that makes
@@ -42,12 +45,23 @@ from server behavior. `v0.2.0` cashed that: the HTTP backend passes the `v0.1.0`
 boundary suite **unchanged**, against an independent oracle, and separately
 against 18 hostile-server behaviors on a real socket.
 
-It also makes this project's first performance claim, and it is a counter on a
+It also made this project's first performance claim, and it is a counter on a
 named fixture rather than a sentence: **a bounded query moved 324 KiB of a
 128 MiB asset — 0.0025 of it — and every byte moved was a byte the caller asked
-for.** The record is
-[docs/reference/BASELINE.md](docs/reference/BASELINE.md), and `amplification` is
-exactly 1.000000 because there is nothing yet to over-fetch.
+for.**
+
+`v0.3.0` is the release that changes those numbers on purpose, and it changes
+them in both directions. Seventeen clustered reads of a header and an index went
+from 18 requests to 3; eight parallel readers went from 152 to 25 and now move
+one reader's worth of bytes between them; the full sequential read did not move
+at all. The bounded query's `selectivity` got *worse*, 0.0025 to 0.0112, because
+alignment converts request count into transferred bytes, and 1191936 bytes of
+what it moved are `bytesOverFetched` — reported beside the saving rather than
+instead of it. Both
+records are counters on a named fixture:
+[BASELINE.md](docs/reference/BASELINE.md) for what the shipped configuration
+costs, and [BLOCK_POLICY.md](docs/reference/BLOCK_POLICY.md) for why it is that
+configuration.
 
 What the tree actually does is in
 [docs/reference/CAPABILITY_MATRIX.md](docs/reference/CAPABILITY_MATRIX.md); what

docs/README.md

@@ -10,14 +10,17 @@ the summary is a documentation bug. When a summary disagrees with
 workspace contract wins; structural changes must update that contract first.
 
 This repository is at `v0.2.0`, released 2026-08-20; its
-[record](releases/v0.2.0.md) states what shipped and what did not. The read
-contract, the local backend, the shared boundary suite, the hostile-server
-corpus, the HTTP backend, and the `ArResolver` bundle are implemented, and a
-`UsdStage` opens over HTTP; no cache is. What the tree actually contains is
-stated in [reference/CAPABILITY_MATRIX.md](reference/CAPABILITY_MATRIX.md), and
-what a bounded query costs is in
-[reference/BASELINE.md](reference/BASELINE.md); everything else here is contract
-and plan.
+[record](releases/v0.2.0.md) states what shipped and what did not. `v0.3.0` is
+in the tree and unreleased: the block cache, its measured constants, and the
+resolver that now decorates every asset it opens. The read contract, the local
+backend, the shared boundary suite, the hostile-server corpus, the HTTP backend,
+the `ArResolver` bundle, and the cache are implemented, and a `UsdStage` opens
+over HTTP. What the tree actually contains is stated in
+[reference/CAPABILITY_MATRIX.md](reference/CAPABILITY_MATRIX.md), what a bounded
+query costs is in [reference/BASELINE.md](reference/BASELINE.md), and why the
+cache's constants are what they are is in
+[reference/BLOCK_POLICY.md](reference/BLOCK_POLICY.md); everything else here is
+contract and plan.
 
 | Category | Answers | Start here |
 | --- | --- | --- |
@@ -60,7 +63,11 @@ and plan.
 - [reference/BASELINE.md](reference/BASELINE.md) is the current recorded I/O
   baseline: the five scenarios METRICS.md §6 requires, the fixture they ran
   against, and what is asserted rather than merely reported. A release record
-  copies it at its tag; it is rewritten whenever I/O behavior changes.
+  copies it at its tag; it is rewritten whenever I/O behavior changes. From
+  `v0.3.0` it holds every scenario twice, with the cache and without it.
+- [reference/BLOCK_POLICY.md](reference/BLOCK_POLICY.md) is the measurement that
+  chose the cache's block size and coalescing gap, the reasoning from it, and
+  the two constants it labels as bounds rather than tuned values.
 
 ## The one-sentence contract
 

docs/architecture/ASSET_READER.md

@@ -151,6 +151,16 @@ and not a consistency feature layered on later:
 | Backend, on mismatch | Fail the read with `AssetChanged`; never rebind, never retry into the new revision |
 | Cache (`v0.3.0`) | Key on the validator, so an entry from revision A cannot serve revision B |
 
+The first row of that table is where the word *observes* earns its place. A
+reader that answers from bytes it captured under its own binding — a block cache
+serving a hit — observes nothing, and what it returns is the revision it is
+bound to. That is the guarantee holding, not an exception to it: the failure
+§2.1 exists to prevent is one reader composing bytes from two revisions, and a
+reader that never leaves revision A cannot. `AssetChanged` is reported by the
+layer that reaches the transport, on the reads that reach it. The boundary suite
+states both halves; see
+[BOUNDARY_SUITE.md](../contributing/BOUNDARY_SUITE.md) §3.
+
 A backend that cannot obtain a usable validator is still bound for its
 lifetime — see §7.3 — but the binding is best-effort, and it says so through
 `IdentityStability::Unavailable`.

docs/architecture/CACHE.md

@@ -4,13 +4,28 @@ This document fixes block caching, request coalescing, single-flight
 de-duplication, and cache identity. It is the contract for `usdAssetCache`,
 which is a decorator over `AssetReader` and knows no transport concept.
 
-Status: planned for `v0.3.0`, with optional persistence in `v0.4.0`. Nothing
-here is implemented.
+Status: implemented in `libs/usd-asset-cache` as of `v0.3.0`, except §8, which
+is `v0.4.0`. The block model, coalescing, single-flight, the key, and eviction
+are in the tree and tested; the constants in §3 and §4 are measured and the
+measurement is [BLOCK_POLICY.md](../reference/BLOCK_POLICY.md); the counters in
+§9 populate and are recorded in [BASELINE.md](../reference/BASELINE.md).
 
 The architecture below is unchanged by the `v0.2.0` reordering; what changed is
-that the validator it keys on already exists when the cache lands. There is no
+that the validator it keys on already existed when the cache landed. There is no
 interim URL-keyed cache and no migration away from one.
 
+Two things the implementation makes more specific than this document did, and
+both are narrower rather than wider:
+
+- **A read *at least* as large as the bypass threshold bypasses**, where §3 says
+  "larger than". The boundary had to fall somewhere and it falls on the side
+  that caches less.
+- **Sharing an entry between two readers requires a strong validator.** §6 keys
+  every entry on the validator, and §8 makes strength the test for outliving a
+  reader; the implementation applies the same test one level earlier, to
+  outliving *this* reader. A weak or absent validator still caches, privately,
+  for the reader's lifetime, which is what §6 promises.
+
 ## 1. Which cache this is
 
 Two caches exist in the whole system, and confusing them is the failure this
@@ -45,7 +60,7 @@ the cache is a decorator: the local backend is used without it.
 ## 3. Block model
 
 ```text
-blockSize        a power of two, default chosen by measurement in v0.3.0
+blockSize        a power of two; 65536 by default, chosen by measurement
 blockIndex       offset / blockSize
 blockRange       [blockIndex * blockSize, (blockIndex + 1) * blockSize)
 ```
@@ -81,6 +96,13 @@ Both numbers are recorded with the measurement that produced them, per
 [METRICS.md](METRICS.md). A tuned constant without a recorded measurement is a
 guess with a decimal point.
 
+They are, and the record is [BLOCK_POLICY.md](../reference/BLOCK_POLICY.md): a
+maximum gap of one block and a maximum merged length of 8 MiB. The record also
+says the thing a tuning document is most tempted to leave out — that at the
+shipped block size the gap does not currently bind on any measured pattern. It
+is 1 because that is the entire measured benefit where the benefit exists, and
+because 2 and 4 were measured and bought nothing anywhere.
+
 ## 5. Single-flight
 
 Concurrent readers that miss the same block issue **one** request. The second
@@ -115,6 +137,12 @@ other validator question belongs to the backend, per §7.1 of
 construct it has become an HTTP cache, and the local backend stops being a
 usable oracle for the cached path.
 
+The key's identity half — identifier, validator, block size — is interned by the
+store, so a per-block lookup costs two integers rather than two string
+comparisons. That is an implementation detail of where the strings live and not
+of what the key is: two readers whose identities compare equal share entries,
+and two whose identities differ never do, which is what this section is about.
+
 The rule that follows is the whole point:
 
 ```text
@@ -131,11 +159,20 @@ reader.
 ## 7. Eviction
 
 - The cache has a bounded memory budget. It never grows to the asset size.
+  128 MiB by default.
 - Eviction is LRU by default, per process, with the budget shared across
   assets so one enormous asset cannot starve the rest of the stage.
 - Eviction is invisible to correctness. An evicted block is re-fetched; it is
   never served stale, and never served zero-filled.
 
+The implementation stripes the store, and the eviction order is therefore LRU
+within a stripe rather than globally. That is a consequence of §5's ban on a
+global lock — a store-wide LRU order needs a store-wide lock on every hit — and
+it is admissible precisely because of the third rule above: eviction is
+invisible to correctness, so an approximate order costs a re-fetch and nothing
+else. The stripe count falls back toward one for a small budget, which makes the
+order exact where a test can see it.
+
 ## 8. Persistence — Planned (`v0.4.0`)
 
 An on-disk cache is admitted only after validators land, because a persistent
@@ -176,3 +213,15 @@ saved by coalescing, requests saved by single-flight, and evictions.
 
 These are not diagnostics. They are the evidence for the project's central
 claim, and a cache change without a before-and-after number is not reviewable.
+
+`v0.3.0` records its before and after in one table:
+[BASELINE.md](../reference/BASELINE.md) measures every scenario twice, with the
+cache and without it, in one run of one harness.
+
+The counters are per reader, and a decorated stack has one reader as far as the
+counters are concerned — the outermost. The two ends of a stack disagree about
+what `bytesRequested` means, so the decorator keeps what the caller asked for
+and takes from the reader underneath only what crossed the transport
+(`ReaderMetrics::AbsorbTransport`). A stack that folded both would compute
+`amplification` over a denominator that is two different measurements added
+together.

docs/architecture/METRICS.md

@@ -8,15 +8,28 @@ lifetime and process aggregate in §3, and the environment-keyed dump in §5 are
 implemented in `libs/usd-asset-io` (`usdAssetIo/Metrics.h`) and populated by both
 backends. The HTTP counters populate, including the requests issued by validator
 capture and by conditional range requests, and `retryCount` and `redirectCount`
-are asserted from tests rather than assumed. The cache counters in §2.2 are
-defined and stay at zero until `v0.3.0`.
+are asserted from tests rather than assumed. The cache counters in §2.2 populate
+from `v0.3.0`, out of `libs/usd-asset-cache`.
+
+A note the cache made necessary. Counters are per reader, and a decorated stack
+has one counter set as far as this document is concerned: the outermost
+reader's. The two ends of a stack disagree about what `bytesRequested` means —
+to the cache it is what the caller asked for, to the reader underneath it is
+what the cache asked for expanded to whole blocks — so the outer set keeps the
+caller's ask and the cache's service, and takes from the inner set only what
+crossed the transport. A stack that folded both would compute `amplification`
+over a denominator that is two different measurements added together.
 
 The baselines in §6 are recorded. `v0.2.0` is the first release that *could*
 record one — bytes now cross a network — and the fixture that was missing exists:
 `tests/baseline` serves one synthetic asset of 128 MiB, which is where
 `selectivity` starts meaning something and the kilobyte corpus assets stopped.
 The current record is [BASELINE.md](../reference/BASELINE.md); a release record
-copies it at its tag.
+copies it at its tag. From `v0.3.0` it holds each scenario twice, with the cache
+and without it, because the rule below asks a release that changes I/O behavior
+for the values before *and* after and that release is the first to change them
+on purpose. What chose the cache's constants is a second record,
+[BLOCK_POLICY.md](../reference/BLOCK_POLICY.md).
 
 ## 1. Why this is a contract and not a debug feature