feat: Azure Blob Storage channel support - #1
Open
tenzinplatter wants to merge 117 commits into
Open
Conversation
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…channel # Conflicts: # Cargo.lock
…rd cache synthesis to az://
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Azure Blob Storage as a conda channel backend, mirroring the existing S3 support. Everything is gated behind a new
azurecargo feature in each crate (rattler_config,rattler_networking,rattler_index,rattler_upload,rattler-bin).az://{container}/{path}via a newAzureMiddlewarethat rewritesaz://→ HTTPS and signs with reqsign's AzureSigner.azblobOperatorbuilt with onlyaccount+endpoint(noaccount_key/sas_token), so reqsign'sDefaultCredentialProviderchain resolves credentials ambiently (env →az login→ managed identity → workload identity → IMDS).[azure-options.<container>](account+ optionalendpoint-url), keyed by container name, analogous to[s3-options.<bucket>].rattler-index azureandrattler-upload azuresubcommands with--account/--endpoint-urlflags.Authentication needs no stored secrets:
az loginfor local dev, managed/workload identity or SP env vars in CI.Notable design points
AzblobConfigsets onlycontainer/root/account_name/endpoint. A unit test assertsaccount_key/sas_tokenstayNone.index_with_channel_metadatais unchanged (storage-agnostic).Known scope limitation (intentional)
rattler-binregistersAzureMiddlewarewith an empty config map and the index/upload subcommands takeaccount/endpoint_urlas explicit CLI flags. Wiring[azure-options.<container>]from a config file into the middleware/index at runtime is the pixi integration follow-up (separate spec) — the rattler CLI layer has no config file. Containers fetched viarattler-bindirectly must therefore use the default{account}.blob.core.windows.netendpoint.Test Plan
azure_opt_tests(34), index lib incl.azblob_config(2), networkingazure_middlewarerewrite/passthrough (4)cargo buildfor each crate with--features azure; whole workspace builds with azure off (gating confirmed)cargo clippy --features azure -- -D warningsclean for networking/index/uploadazure_azurite.rs,#[ignore]) compiles; run with a local Azurite +AZURE_STORAGE_ACCOUNT_KEYto exercise the write/list/read path