perf: bound archive cache transport and preserve v1 write compatibility

[Iviesever]

Final status: merged with an explicit integrity/performance tradeoff

Merged as 6f24bb4d2f5e323eb7b3823c3acf367dcccf9446 after the owner's delegated review. The previous rebuild-performance hold is closed. VERSION remains 5.4.0; no tag, changelog or GitHub Release publication.

• Original measured head: bf70ed1024ab37ced2393e73edc4ed8e3c4868bb.
• Up-to-date integration head: 36edabdcf2e2e50af41c77c50de2a58102a50ebb, which includes main 4bab036f8ca8b2982acaa8969572329ceba5579a without changing source bytes.
• Original head, integration head and merged main have identical tree 00c5b8e600ca188d3ce6158f002390307ee3c887.
• Seven files, no new product translation unit. Rejected perf: bound link and discovery cache reads to opened streams #158 remains excluded.

At merge time the latest required Native C++ #683/documentation checks had passed, and this exact source tree had already passed full Native Release/self-host/package #583. Repeat integration Release #584 was still executing at that moment; it subsequently completed successfully. No rule was bypassed, and that chronology is retained rather than rewriting the earlier gate status.

Accepted implementation and compatibility boundary

Archive cache reads now use the unchanged shared bounded reader: one binary stream, same-stream length/read, 64 MiB pre-allocation bound, exact read and observed-growth/EOF-error checks. The v1 quoted-text decoder reads owned bytes through a read-only C++23 span stream, without a second whole-payload copy.

The writer serializes unchanged v1 bytes into a bounded private buffer before creating directories or touching the temporary/destination file. Escaped/delimiter bytes count against the same 64 MiB bound. Oversized saves preserve previous cache contents and unused destination directories. Exactly 64 MiB is accepted; one extra byte is rejected. The 100000-object limit, field/order semantics, accepted whitespace and normal replacement/cleanup remain.

The new total-byte limit intentionally rejects older oversized archive records. The unchanged coordinator warns/re-archives; a too-large new record cannot be cached but does not invalidate an otherwise successful build. This is not a total-process-memory limit or an atomic snapshot against arbitrary concurrent in-place or timestamp-preserving modification.

No archive signature, object freshness, librarian invocation, retry, scheduling, output or other loader changed. Historical write instrumentation opened(0) remains; cache-read work includes decoding and is not pure elapsed I/O share.

Final correctness gates

• Native C++ #683 / 34182430480: success, including Debug jobs and native policy/freshness checks.
• Native Release #584 / 34182430523: complete success, all four Release jobs and self-host/package job 101925809405, including runtime-only staging, validation and artifact upload. publish-release skipped.
• Earlier same-tree Native C++ #681 and full Release/package #583 remain successful historical gates.
• Archive Cache Evidence refactor: add first-class incremental link state #2 / 34182430477: success; original feat: make mqb CLI compile one TU incrementally #1 also passed all corruption/reuse contracts.
• Performance Evidence #517 / 34182430506: successful collection of the unchanged general harness, not a blanket performance pass. Readiness-triggered runs are separate and are not the fixed decision experiment.
• Documentation, compilation database, build plan, cross-stage, incremental, PCH and module checks passed on the integration head.

The 79-test native inventory includes legacy std::quoted byte oracles, 256 deterministic binary-stamp cases, Unicode/space paths, malformed/count/version/EOF checks, exact-limit read/write, escaped overflow, old-cache preservation and no temporary residue. Real-build damage cases require an archive miss without TU recompilation, followed by a genuine no-write/no-tool hit. No freshness assertions were weakened.

Fixed follow-up that resolves the decision

Archive Merge Decision #2 / run 34182041180, artifact 10039349724.

Verified ZIP SHA-256: f71e87dc5292218d51d181756b6d68ae522be44f4a839641f6f02b9e43d13061.

Exact main 4bab036... versus original candidate bf70ed1..., one Windows runner, fixed 40 alternating single-TU rebuild pairs with timings OFF. A/B writes use equal source bytes/length within a pair and actual later timestamps, never manufactured future timestamps. Each result contains exactly one compile and one archive. Post-rebuild audits require full hits, no tools/writes, equal semantic counters and unchanged cache/artifact metadata. Per-audit inventories and raw transcripts are retained.

Measurement	Paired median delta	Paired median percent	MAD	Observed P95	Faster pairs
static129 single-TU rebuild, timings OFF	-0.42615 ms	-0.34397%	3.7734 ms	+7.6024 ms	21/40
32 real public cache-save calls per batch	-0.5542 ms/batch	-1.49076%	3.72545 ms	+70.0751 ms	14/24

The save probe measures serialization plus real temporary-file replacement, not isolated formatting or external process time. The batch median difference divided by 32 is about -0.0173 ms per call; it is not a measured median of individual call durations. Probe and rebuild clocks are not interchangeable. All 322 recorder transcript hashes, 48 probe outputs, exact identities and paired statistics were independently checked.

The predeclared practical rebuild flag (median above both 5 ms and 3%, with at least 30/40 slower pairs) was not triggered. That does not prove zero regression or characterize production tails. The fixed experiment is close to neutral and does not reproduce a large typical cost; actual save API batches likewise show no large typical increase. This supports accepting the bounded persistence contract, not advertising speedup.

The first harness attempt, run 34181612981, failed before measurements because historical seed v5.0.0-rc.2 interpreted modern build as a source filename. Only that harness invocation was corrected. Product heads, probe source, sample counts and decision rules were unchanged. No favorable-only rerun was used.

Earlier adverse and integration evidence remains separate

Original Archive Cache Evidence #1: run 34148454489 / artifact 10028667979, digest 762eeb16fa9684d9fee8fa6a8ff04b36a12c8692ffdb5979b192dcaca61675d7. All 204 pairs remain, including timings-ON single-TU +38.1584 ms / +22.33%, 1/6 faster, cold +92.0604 ms / +2.15%, and approximately neutral large warm samples. Read/write attribution cannot explain away that total difference; no OS cause was proved.

Automatic integration Archive Evidence #2: run 34182430477 / artifact 10039492780, digest a5ecdb5e7b368b9997e2f46a1b95407edbaa598bc866f1c1459bc68cf0244ce3. Same source trees, exact recorded main/integration SHAs, all 204 pairs and 908 transcript hashes independently checked. Single-TU timings-ON is -4.2411 ms / -4.1856%, 4/6 faster, but P95/max is +50.3611 ms. Cold median is -4.8877 ms / -0.0769% with a +6295.0658 ms maximum. Large warm timings-OFF default is -0.0690 ms / -0.3113%, verbose +0.1506 ms / +0.6773%, and timings-ON default +0.3312 ms / +1.4618%. These mixed results are retained, not pooled with the fixed timings-OFF decision or used to prove an OS explanation.

Original general Performance #512 artifact 10028705604, digest 67d71399687613a0d9d67fafff8390158b3922f55111ea26665bb372feacf07d, remains an unaffected-path control: its instrumented executable/module scenarios have zero archive payload opens and zero librarian launches. Favorable controls are not attributed to this patch or used to cancel its static rebuild observations.

Remaining route

The decision is accepted and merged, not another indefinite hold. #161 independently binds toolchain v9 decoding to its existing byte limit while retaining all trust checks. It must not repeat #160's attribution. #158 is still rejected. Object handoff remains freshness-gated; cache packs require isolated residual I/O evidence; residency remains v6.0. Final cumulative performance must compare the accepted combination against released v5.4.0, followed by a separate VERSION/changelog/release-preparation PR.

[Iviesever]

Final performance decision: accept, subject to normal integration gates

The owner delegated the decision. Accept #160 for its bounded read/write integrity contract, not for a claim of warm/cold acceleration. Do not keep the previous six-sample rebuild hold open indefinitely.

The fixed follow-up used exact main 4bab036f8ca8b2982acaa8969572329ceba5579a and original candidate bf70ed1024ab37ced2393e73edc4ed8e3c4868bb, one Windows runner, 40 alternating timings-OFF single-TU rebuild pairs, identical source bytes/length per pair and real timestamp advancement. Every rebuilt result has exactly one compiled TU and one archive; subsequent instrumented audits retain full hits, no tools/writes, identical semantic counters and unchanged artifact/cache metadata. Full per-audit inventories are retained.

• Run: https://github.com/Iviesever/msvc-quick-build/actions/runs/34182041180
• Artifact: https://github.com/Iviesever/msvc-quick-build/actions/runs/34182041180/artifacts/10039349724
• Verified ZIP SHA-256: f71e87dc5292218d51d181756b6d68ae522be44f4a839641f6f02b9e43d13061
• 40 rebuild pairs: paired median -0.42615 ms / -0.34397%, MAD 3.7734 ms, observed P95 +7.6024 ms, 21/40 faster. The predeclared practical adverse flag is false; this does not prove no regression.
• 24 pairs of 32 real public ArchiveCacheFile::save calls: paired batch median -0.5542 ms / -1.49076%, MAD 3.72545 ms, P95 +70.0751 ms, 14/24 faster. This measures serialization PLUS real file replacement, not pure formatting and not full-process elapsed. Per-call median batch difference is about -0.0173 ms; its outliers are retained.
• Independently recomputed statistics, all 322 recorded invocation transcript hashes, pass marker and exact SHA identities were checked.

The old instrumented six-pair +38.1584 ms / +22.33% rebuild observation is retained, not pooled with this timings-OFF comparison or dismissed as a proven OS effect. This follow-up does not reproduce a material typical cost in the focused path, and the actual save API shows no large typical increase. Together with the byte-limit correctness tests and previous full native/package validation, the boundary improvement is worth accepting. No claim of universal speedup or causal exoneration is made.

First attempt 34181612981 failed before measurements because the historical v5.0.0-rc.2 seed does not accept the modern build subcommand. Only that harness invocation was corrected; product heads, probe source and decision rules were unchanged. This is the first completed follow-up, not a favorable-only rerun.

Current integration head 36edabdcf2e2e50af41c77c50de2a58102a50ebb adds current main to ancestry with the identical candidate tree 00c5b8e600ca188d3ce6158f002390307ee3c887. Required checks are rerunning normally; no repository rule is bypassed. VERSION is still 5.4.0 and no release/tag change is authorized here. The next isolated theme is bounded toolchain-cache decoding with the existing trust/age/PATH checks untouched.