Azure metadata cache hookup and Azure integration test updates

[zsy056]

Azure metadata cache hookup and Azure integration test updates

Summary

This PR wires duckdb-azure into DuckDB HTTPFS-style metadata caching and restores Azure multi-file auto-glob behavior for bare Blob directory paths on HNS-backed accounts.

What changed

• Hook Azure file metadata loading (GetProperties() / file info lookup) into HTTPFS-aligned metadata cache behavior.
  • enable_http_metadata_cache = false now uses a query-local cache.
  • enable_http_metadata_cache = true now uses a shared/global cache.
• Seed file metadata from Azure blob/DFS listings so file size and last-modified can be reused without an extra metadata round trip.
• Invalidate Azure metadata cache entries on mutating operations:
  • write/append opens
  • delete/remove
  • create directory
• Use shared_mutex for the shared/global AzureMetadataCache to allow concurrent readers with exclusive writes.
• Make the Blob no-wildcard Glob path return only regular files so HNS directories do not suppress DuckDB's fallback auto-glob for multi-file readers.
• In Blob glob results, only stamp type=file for non-empty items before seeding metadata, so flat HNS Blob listings do not misclassify directory entries as files.
• Update Azure SQLLogicTests:
  • adjust stale HTTP-stats byte expectations in test/sql/azure.test
  • restore test/sql/azure_glob.test to the original auto-glob form (read_csv('az://.../partitioned'))

Why

Before this change, repeated Azure file opens inside a single query were still issuing repeated metadata lookups. On the benchmark parquet workload, that kept Azure HEAD traffic high and wall time far above direct download speed.

The metadata-cache hookup removes those repeated metadata calls while preserving HTTPFS-style semantics:

• query-local reuse by default
• shared/global reuse when explicitly enabled

Benchmark results

Benchmark workload:

• 76.3 MiB uncompressed parquet
• 20 row groups

Primary before/after comparison (same curl + object-cache-on + 4 MB / 4 MB setup):

Metric	Before metadata-cache hook	After metadata-cache hook
real median	47.257 s	14.443 s
throughput	1.61 MiB/s	5.28 MiB/s
#HEAD	22	1
#GET	21	21

Net result:

• about 3.27x faster wall time on this workload
• Azure metadata requests collapsed from 22 HEADs to 1
• These measurements were collected on a dev box and may reflect higher network latency than a storage-adjacent environment; that extra latency can amplify the cost of repeated metadata requests, so the measured speedup may be skewed upward relative to a lower-latency setup.

Post-fix comparison for the two configurations we want to highlight:

Configuration	real (s)	Throughput (MiB/s)	#HEAD	#GET
chunk = 4 MiB, buffer = 4 MiB	14.443	5.28	1	21
chunk = 1 MiB, buffer = 1 MiB	14.194	5.38	1	81

Validation

• Rebuilt locally with MSVC/CMake.
• Final result: 9 test cases, 141 assertions, all passed for the targeted Azure integration subset.

[belgattitude]

Really waiting for this one :)

[zsy056]

Hi @benfleis , wondering if by any chance you can help review this change? Thanks!