fix(lib-storage): replace UploadPart spread with field allowlist - #8029
fix(lib-storage): replace UploadPart spread with field allowlist#8029Zelys-DFKH wants to merge 1 commit into
Conversation
|
I think this still has the same root issue in a slightly narrower form. The PR removes the broad I tested this locally with a temporary regression test using: ChecksumSHA256: "object-level-sha256-base64=="on the That looks like the same class of bug as Because of that, I don’t think the safe boundary is “fields accepted by |
…ing allowlist Upload.__doConcurrentUpload spreads ...this.params (PutObjectCommandInput) into every UploadPartCommand. That carries two classes of fields the UploadPart API can't handle: 1. PUT-only fields S3 rejects with NotImplemented (501) or MalformedXML: IfNoneMatch, ContentType, Metadata, ACL, StorageClass, ObjectLock*. 2. Object-level checksums whose value is a whole-object hash and is wrong as a per-part value: ContentMD5 and the precomputed Checksum* family (ChecksumSHA*/CRC*/CRC64NVME). Sending them as per-part values causes BadDigest (aws#6742). Replace the spread with an explicit allowlist of fields accepted by UploadPart with the same semantic meaning as on PutObject: Bucket, Key, ChecksumAlgorithm, SSECustomerAlgorithm/Key/KeyMD5, RequestPayer, ExpectedBucketOwner. ChecksumAlgorithm is intentionally kept; it is a directive (e.g. "SHA256"), not a precomputed hash, so the SDK checksum middleware can still compute a per-part value from it. Use inline this.params.X references instead of destructure-then-rebuild: avoids shadowing the imported ChecksumAlgorithm enum at module scope and halves the surrounding LOC. CompleteMultipartUploadCommand continues to receive ...this.params, so IfNoneMatch and other CompleteMultipartUpload-valid fields still work. Regression coverage: a multipart test asserts IfNoneMatch, ContentType, ContentMD5, and ChecksumSHA256 don't reach UploadPart and that IfNoneMatch still reaches CompleteMultipartUpload; a single-part PUT path test confirms object-level fields are preserved when multipart is not used. All 54 tests pass. Fixes aws#8020. Partially addresses aws#6742. Credit: @danyalahmed1995 spotted the Checksum* leak on review.
ad69e64 to
22fc2bc
Compare
|
Good catch. You're right that the allowlist still leaks object-level checksums into UploadPart. Pushed This complements #7990 (fail-fast on full-object |
Issue
#8020
Description
Upload.__doConcurrentUploadspreads...this.params(typed asPutObjectCommandInput) into everyUploadPartCommand.PutObjectCommandInputincludes object-creation-only fields —IfNoneMatch,ContentType,ContentMD5,Metadata,ACL,StorageClass,ObjectLock*, and others — and S3 rejects them on the UploadPart API withNotImplemented(501) orMalformedXML.This PR replaces the spread with an explicit allowlist of the fields
UploadPartRequestactually accepts:Bucket,Key, the fullChecksum*family,SSECustomerAlgorithm/Key/KeyMD5,RequestPayer, andExpectedBucketOwner.Body,PartNumber,UploadId, andContentLengthare set explicitly as before.Two intentional omissions:
ContentMD5—this.params.ContentMD5is the whole-object MD5. Forwarding it to each part would be wrong. TheUploadPartRequest.ContentMD5field exists for per-part hashes, which callers can supply via request interceptors.ContentLength— already explicitly set toundefinedin the existing code; no change.CompleteMultipartUploadCommandcontinues to receive...this.paramsat the end of the flow, soIfNoneMatchand otherCompleteMultipartUpload-valid fields still work correctly there.Note on PR #7994: That PR strips
ContentMD5via a denylist. This PR uses a full allowlist instead — more defensive against futurePutObjectCommandInputadditions silently leaking through. Happy to coordinate if the team prefers the denylist approach.Testing
Added a regression test in the
large buffersdescribe block that:IfNoneMatch,ContentType, andContentMD5don't reachUploadPartCommandSSECustomerAlgorithm(valid for UploadPart) flows throughCompleteMultipartUploadCommandstill receivesIfNoneMatchAll 53 existing tests pass.
Checklist
*.integ.spec.ts) or E2E tests.@public,@internaltags and enabled doc generation on the package. Remember that access level annotations go below the description, not above.