delta_scan silently ignores workload_identity in Azure credential chain, falls back to IMDS

[MCBoarder289]

Summary
When using delta_scan with an Azure secret configured as CHAIN 'workload_identity', DuckDB falls through to the managed identity IMDS endpoint instead of using the federated token. This pattern is common when federating credentials in AKS, and I am experiencing this currently in an AKS environment.

A similar issue/symptom was previously reported in #64 and closed without a fix, as the credential bridge for non-cli chain types sets no options on the underlying object_store builder and relies on delta-kernel-rs independently resolving the same credentials, which doesn't work for the workload_identity pattern.

Root Cause
I came across this because read_parquet worked just fine, but delta_scan didn't, and the difference was in how those two functions resolve Azure auth:

• read_parquet - duckdb-azure → Azure C++ SDK (WorkloadIdentityCredential reads env vars natively)
• delta_scan - duckdb-delta → delta-kernel-rs → object_store crate (deferred to rust crate implementation)

In src/functions/delta_scan/delta_multi_file_list.cpp, CreateBuilder() handles the credential_chain secret type. For anything that isn't cli, the code sets zero options on the object_store builder and falls back to a "hope for the best" comment:

 } else if (provider == "credential_chain") {
     if (chain.find("cli") != std::string::npos) {
         set_option(builder, "use_azure_cli", "true");
     }
     // non-cli credential chains will just "hope for the best"...
 }

The object_store crate's MicrosoftAzureBuilder is constructed with new() (not from_env()), so it never reads AZURE_FEDERATED_TOKEN_FILE, AZURE_CLIENT_ID, or AZURE_TENANT_ID from the environment. With none of those three set, resolve_credential_auto() skips the workload identity branch and falls all the way through to:

 } else {
     self.resolve_managed_identity(http)
 }

How to Reproduce

 CREATE SECRET azure_wi (
     TYPE AZURE,
     PROVIDER CREDENTIAL_CHAIN,
     CHAIN 'workload_identity',
     ACCOUNT_NAME 'mystorageaccount'
 );

• Fails with: "Error performing token request" / IMDS timeout at 169.254.169.254
  SELECT * FROM delta_scan('abfss://container@mystorageaccount.dfs.core.windows.net/path/to/table');

• Works fine (different auth stack):
  SELECT * FROM read_parquet('abfss://container@mystorageaccount.dfs.core.windows.net/path/to/file.parquet');

• Expected error:

   Error interacting with object store: Generic MicrosoftAzure error: Error performing token request:
   Error after 10 retries in 1.9s, source: error sending request for url
   (http://169.254.169.254/metadata/identity/oauth2/token?...)
  

Proposed Fix
Forward the three workload identity environment variables as explicit options on the delta-kernel-rs builder, so object_store's resolve_credential_auto() takes the correct branch:

   if (chain.find("workload_identity") != std::string::npos) {
       const char* fed_token = getenv("AZURE_FEDERATED_TOKEN_FILE");
       const char* azure_client_id = getenv("AZURE_CLIENT_ID");
       const char* azure_tenant_id = getenv("AZURE_TENANT_ID");
       if (fed_token) set_option(builder, "federated_token_file", fed_token);
       if (azure_client_id) set_option(builder, "azure_client_id", azure_client_id);
       if (azure_tenant_id) set_option(builder, "azure_tenant_id", azure_tenant_id);
   }

With those three options set, resolve_credential_auto() in object_store will reach the workload identity path

I have tested this locally and it works, and will put up a PR as a follow-up to this issue.

Workaround
Currently in k8s, I'm working around this by manually injecting service principal credentials via Kubernetes secrets and creating a service_principal secret in DuckDB.

Federated credentials in k8s don't have secrets, so this is an anti-pattern when trying to run it in AKS.