Skip to content

STG 103/104 Photon Phase 1 + Blob Access Tier + MD5/CRC64 Combined Return#39234

Open
aanubhav-msft wants to merge 14 commits into
feature/storage/STG_103_104from
feature/storage/STG104_Blob_Files
Open

STG 103/104 Photon Phase 1 + Blob Access Tier + MD5/CRC64 Combined Return#39234
aanubhav-msft wants to merge 14 commits into
feature/storage/STG_103_104from
feature/storage/STG104_Blob_Files

Conversation

@aanubhav-msft

Copy link
Copy Markdown
Member

Packages impacted by this PR

  • @azure/storage-blob (bumped to 12.34.0-beta.1)
  • @azure/storage-common

Issues associated with this PR

Describe the problem that is addressed by this PR

This PR upgrades @azure/storage-blob to target the Storage service version 2026-10-06 and adds several new capabilities exposed by that version:

  1. Apache Arrow list response formatlistBlobsFlat and listBlobsByHierarchy can now optionally return results in the Apache Arrow columnar format instead of XML, which is more efficient for large blob listings. Callers opt in via responseFormat: StorageResponseFormat.Arrow. The default remains XML for full backward compatibility.
  2. New endBefore list option — allows bounding a listing on the upper end (Apache Arrow path only).
  3. Access-tier metadata on blob downloadaccessTier, accessTierInferred, accessTierChangedOn, and smartAccessTier are now surfaced on the blob download response.
  4. CRC64 checksum on upload operations — for service version 2026-10-06+, upload operations return the service-computed CRC64 (xMsContentCrc64) in addition to contentMD5 when a Content-MD5 is provided.

What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?

Are there test cases added in this PR? (If not, why?)

Yes:

  • test/node/containerClientListBlobsApacheArrow.spec.ts — new suite covering flat/hierarchy listing via Apache Arrow, including properties, metadata, tags, paging, prefixes, snapshots, versions, soft-deleted blobs, special characters, and startFrom/endBefore bounds.
  • test/node/blockblobclient.spec.ts — new "MD5/CRC64 combined return" suite verifying xMsContentCrc64 is returned for stageBlock, appendBlock, uploadPages, stageBlockFromURL, appendBlockFromURL, syncUploadFromURL, and uploadPagesFromURL.
  • test/blobclient.spec.ts — new assertions for access-tier properties on the download response.

Provide a list of related PRs (if any)

Command used to generate this PR:**(Applicable only to SDK release request PRs)

Checklists

  • Added impacted package name to the issue description
  • Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
  • Added a changelog (if necessary)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

  • Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Pull request overview

Updates the Storage Blob SDK to align with Storage service version 2026-10-06, introducing an opt-in Apache Arrow listing format, surfacing additional blob download access-tier headers, and returning service-computed CRC64 alongside MD5 for certain upload operations.

Changes:

  • Added StorageResponseFormat (Auto/Xml/Arrow) in @azure/storage-common and re-exported it from @azure/storage-blob to opt into Apache Arrow blob listings.
  • Implemented Apache Arrow list response parsing with an XML fallback path, plus support for the new endBefore list option (Arrow-only).
  • Surfaced access-tier metadata on download responses and updated upload operation response headers to include xMsContentCrc64 (service version 2026-10-06+).

Reviewed changes

Copilot reviewed 24 out of 34 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
sdk/storage/storage-common/src/StorageResponseFormat.ts Introduces StorageResponseFormat enum for list response selection.
sdk/storage/storage-common/src/indexPlatform.ts Exports StorageResponseFormat from the node platform entrypoint.
sdk/storage/storage-common/src/indexPlatform-browser.mts Exports StorageResponseFormat from the browser platform entrypoint.
sdk/storage/storage-common/review/storage-common-react-native.api.diff.md Updates API diff for react-native surface (line shifts due to new exports).
sdk/storage/storage-common/review/storage-common-node.api.md Adds StorageResponseFormat to the public node API surface.
sdk/storage/storage-common/review/storage-common-browser.api.diff.md Updates API diff for browser surface (line shifts due to new exports).
sdk/storage/storage-common/CHANGELOG.md Documents the new StorageResponseFormat feature.
sdk/storage/storage-blob/test/node/containerClientListBlobsApacheArrow.spec.ts Adds test coverage for Arrow-based flat/hierarchy listings and options.
sdk/storage/storage-blob/test/node/blockblobclient.spec.ts Adds tests validating MD5 + returned CRC64 behavior across upload operations.
sdk/storage/storage-blob/test/blobclient.spec.ts Adds tests for access-tier properties on download responses.
sdk/storage/storage-blob/swagger/README.md Updates autorest input/spec version and service api-version override to 2026-10-06; adds pageable-hiding directives for Arrow list paths.
sdk/storage/storage-blob/src/utils/utils.common.ts Adds resolveResponseFormat helper and imports StorageResponseFormat.
sdk/storage/storage-blob/src/utils/constants.ts Bumps SDK/service version constants and adds ApacheArrowContentType.
sdk/storage/storage-blob/src/utils/blobListArrowParser.ts Adds Arrow IPC parsing and XML fallback deserialization helpers.
sdk/storage/storage-blob/src/index.ts Re-exports StorageResponseFormat from @azure/storage-common.
sdk/storage/storage-blob/src/generatedModels.ts Preserves SyncCopyStatusType via a compatibility alias.
sdk/storage/storage-blob/src/generated/src/storageClient.ts Updates generated UA string and default service version constant.
sdk/storage/storage-blob/src/generated/src/operationsInterfaces/container.ts Adds generated Apache Arrow list operation signatures.
sdk/storage/storage-blob/src/generated/src/operations/pageBlob.ts Updates generated Accept parameter reference for page blob upload.
sdk/storage/storage-blob/src/generated/src/operations/container.ts Adds generated Arrow list operation specs returning raw stream bodies.
sdk/storage/storage-blob/src/generated/src/operations/blockBlob.ts Updates generated Accept parameter reference for block blob ops.
sdk/storage/storage-blob/src/generated/src/operations/appendBlob.ts Updates generated Accept parameter reference for append blob ops.
sdk/storage/storage-blob/src/generated/src/models/parameters.ts Adds Arrow Accept header constant + endBefore; renames prior Accept param to avoid collisions.
sdk/storage/storage-blob/src/generated/src/models/mappers.ts Adds Arrow list headers mappers and new download/upload headers (tier + CRC64).
sdk/storage/storage-blob/src/generated/src/models/index.ts Adds new headers/interfaces and access-tier + CRC64 header properties in generated types.
sdk/storage/storage-blob/src/ContainerClient.ts Adds responseFormat/endBefore options and Arrow listing execution + parsing with XML fallback.
sdk/storage/storage-blob/src/BlobDownloadResponse.ts Exposes access-tier fields via typed getters.
sdk/storage/storage-blob/review/storage-blob-node.api.md Updates public API surface for new options/exports/headers.
sdk/storage/storage-blob/review/storage-blob-browser.api.diff.md Updates browser vs node API diff for new exports/types.
sdk/storage/storage-blob/package.json Bumps version and adds runtime dependency on apache-arrow.
sdk/storage/storage-blob/CHANGELOG.md Documents service version bump and new Arrow/tier/CRC64 capabilities.
sdk/storage/storage-blob/assets.json Updates assets tag for recordings.
eng/approved-third-party-dependencies.yml Adds an exception allowing apache-arrow for @azure/storage-blob.
Files excluded by content exclusion policy (1)
  • pnpm-lock.yaml

Comment thread sdk/storage/storage-blob/src/ContainerClient.ts Outdated
Comment thread sdk/storage/storage-blob/src/ContainerClient.ts Outdated
Comment thread sdk/storage/storage-common/CHANGELOG.md
Comment thread sdk/storage/storage-blob/CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants