feat: bound CAS downloads with per-file size caps (#31)#37
Merged
Conversation
Sidetree caps the size of each anchored file (core index 1MB, core/provisional
proof 2.5MB shared, provisional index 1MB, chunk 10MB) and bounds gzip
expansion to MaxMemoryDecompressionFactor (3x) as a zip-bomb guard. Our reader
enforced none of it: fetchCoreIndexFile/CoreProof/ProvisionalIndex/Provisional
Proof/Chunk called cas.Get unbounded, and two `// TODO Check Max ... File Size`
markers (core_index.go, chunk.go) were never resolved. A malicious or corrupt
CID could expand into an unbounded in-memory parse.
Thread the per-file cap into the download the way the reference
DownloadManager.download(uri, maxSizeInBytes) does:
- CAS.Get gains a maxSizeInBytes parameter. Its contract: bound the
(compressed) download to maxSizeInBytes and bound decompression to
maxSizeInBytes * MaxMemoryDecompressionFactor; oversized content is
permanently invalid (immutable CID), so return an ErrMalformed-class error.
- Each fetch* method passes the protocol cap for the file type it fetches and
defensively re-checks the returned (decompressed) length against
cap * MaxMemoryDecompressionFactor (the maximum legal decompressed size), in
case a CAS does not honor the contract. This bound never rejects a valid
anchor — it is exactly ION's decompression ceiling.
Oversized files route through classifyMalformed -> ErrMalformed (permanent skip).
New sentinel ErrFileTooLarge. Resolves the two file-size TODOs.
Downstream: ion-node's CAS adapter must add the maxSizeInBytes parameter to Get
at its next sidetree-go repin (#53) and should enforce the bounded download +
3x decompression cap there (the CAS owns the compressed stream). ion-node pins a
fixed sidetree-go commit, so its build is unaffected until that repin.
Tested: TestProcessorRejectsOversizedFile (a >cap*3 file -> ErrFileTooLarge,
asserted ErrMalformed) and TestFetchPassesPerFileSizeCaps (a full five-file
batch processes cleanly and each fetch passes the correct per-file cap, captured
by the test CAS). The test/contract CAS implementations adopt the new signature.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sidetree caps each anchored file's size and bounds gzip expansion (zip-bomb guard). Our reader enforced none of it — the five
fetch*methods calledcas.Getunbounded, and two// TODO Check Max … File Sizemarkers (core_index.go,chunk.go) were never resolved. A malicious/corrupt CID could expand into an unbounded in-memory parse.This threads the per-file cap into the download, mirroring the reference
DownloadManager.download(uri, maxSizeInBytes). Resolves both TODOs. Plan:docs/plans/2026-06-04-001-feat-ion-value-locking-protocol-rules-plan.md.Per-file caps (from
params.go, #30)MaxCoreIndexFileSizeInBytes(1 MB)MaxProofFileSizeInBytes(2.5 MB, shared)MaxProvisionalIndexFileSizeInBytes(1 MB)MaxProofFileSizeInBytes(2.5 MB, shared)MaxChunkFileSizeInBytes(10 MB)Change
CAS.Getgains amaxSizeInBytesparameter. Its documented contract: bound the (compressed) download tomaxSizeInBytes, and bound decompression tomaxSizeInBytes × MaxMemoryDecompressionFactor(3×); oversized content is permanently invalid (immutable CID) → return anErrMalformed-class error.fetch*passes the protocol cap for its file type and defensively re-checks the returned (decompressed) length againstcap × MaxMemoryDecompressionFactor— the maximum legal decompressed size — in case a CAS doesn't honor the contract. This defensive bound never rejects a valid anchor: it is exactly ION's decompression ceiling.classifyMalformed→ErrMalformed(permanent skip). New sentinelErrFileTooLarge.ion-node's CAS adapter must add themaxSizeInBytesparameter toGetat its next sidetree-go repin (#53), and should enforce the bounded download + 3× decompression cap there (the CAS owns the compressed stream).ion-nodepins a fixed sidetree-go commit, so its build is unaffected until that repin — no breakage now.Testing
TestProcessorRejectsOversizedFile— a> cap×3file →ErrFileTooLarge, assertedErrMalformed.TestFetchPassesPerFileSizeCaps— a full five-file batch processes cleanly and each fetch passes the correct per-file cap (captured by the test CAS).gofmtclean;go build,go vet,go test -race -count=1 ./...all green.Post-Deploy Monitoring & Validation
ErrMalformedskips carryingErrFileTooLarge("sidetree file exceeds its maximum size").cap×3(ION's own ceiling), so it cannot reject a valid anchor.ErrFileTooLargeon a known-good mainnet anchor would indicate a wrong cap or a decompression-ratio surprise → revert and investigate.🤖 Generated with Claude Code