Enable live catalog with region-aware fallback engine by baijumeswani · Pull Request #789 · microsoft/Foundry-Local

baijumeswani · 2026-06-09T07:06:10Z

Replaces the static, file-based model catalog with a live Azure Foundry catalog client, and makes both catalog queries and model downloads region-aware. The client now detects the caller's Azure region from cluster headers, queries that region's catalog endpoint, and transparently retries against nearby regions when an endpoint is unhealthy. Downloads resolve their SAS URI from the same region that served the model's catalog entry.

This ports three behaviors from the C# core into the C++ SDK:

Fetch the live catalog instead of reading a baked-in static snapshot.
Detect the region from azureml-served-by-cluster headers (instead of always using eastus).
Region-aware catalog query and model download.

How it works

                        ┌──────────────────────────────────────────┐
                        │  AzureModelCatalog.FetchModels()         │
                        └───────────────────┬──────────────────────┘
                                            │
                                            ▼
                        ┌──────────────────────────────────────────┐
                        │  AzureCatalogClient                      │
                        │                                          │
                        │  1. Detect region                        │
                        │     POST probe ─► read                   │
                        │     "azureml-served-by-cluster" header   │
                        │     vienna-<region>-NN ─► region         │
                        │     (explicit config wins; else eastus)  │
                        └───────────────────┬──────────────────────┘
                                            │ for each device filter set
                                            ▼
                        ┌──────────────────────────────────────────┐
                        │  RegionFallback.Execute(startRegion)     │
                        │                                          │
                        │   chain = [start]                        │
                        │         + proximal[start]                │
                        │         + one random public region       │
                        └───────────────────┬──────────────────────┘
                                            │
                       ┌────────────────────┼─────────────────────┐
                       ▼                     ▼                     ▼
                  try region A ─►       try region B ─►       try region C
                   2xx? ✓ done          retryable(0/408/        ...exhausted
                   permanent(404)? ✓     429/5xx)? ─► next       ─► fail set
                   stop                                          (others continue)
                       │
                       │ page 1 pins the serving region;
                       │ later pages stay on it (tokens are region-specific)
                       ▼
            ┌──────────────────────────────────────────┐
            │  Models stamped with detected_region,    │
            │  committed atomically per filter set     │
            └───────────────────┬──────────────────────┘
                                │  (download)
                                ▼
            ┌──────────────────────────────────────────┐
            │  DownloadManager.DownloadModel(info)     │
            │                                          │
            │  region = explicit config                │
            │         → info.detected_region           │
            │         → eastus                         │
            └───────────────────┬──────────────────────┘
                                ▼
            ┌───────────────────────────────────────────┐
            │  ModelRegistryClient.ResolveModelContainer│
            │  (same RegionFallback chain) ─► SAS URI   │
            └───────────────────┬───────────────────────┘
                                ▼
                          Blob download

catalog_region: ""/"auto" → detect from headers; any other value → explicit override.
DisableRegionFallback: disables cross-region retries (single-region behavior).

vercel · 2026-06-09T07:06:16Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview, Comment	Jun 15, 2026 10:28pm

Copilot

Pull request overview

This PR replaces the C++ SDK’s embedded/static model catalog with a live Azure Foundry catalog client, adds automatic region detection from azureml-served-by-cluster, and introduces a region-fallback engine used by both catalog queries and model-registry SAS resolution so downloads target the region that served the catalog entry.

Changes:

Removed the static snapshot catalog + snapshot generation tool, and always build/use the live Azure catalog client.
Added RegionFallback + status-aware HTTP response plumbing to support regional retries and sticky-region behavior.
Stamped models with detected_region and used it to choose the model registry region when resolving download SAS URIs (with configurable fallback disable).

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sdk_v2/cpp/tools/catalog_snapshot/main.cc	Removed the snapshot generation tool entrypoint (static catalog pipeline retired).
sdk_v2/cpp/tools/catalog_snapshot/CMakeLists.txt	Removed CMake target for the snapshot tool.
sdk_v2/cpp/test/sdk_api/shared_test_env.h	Test env now always points at the live Azure catalog URL (no static path).
sdk_v2/cpp/test/internal_api/static_catalog_test.cc	Removed tests that exercised the embedded static catalog client.
sdk_v2/cpp/test/internal_api/region_fallback_test.cc	Added unit tests for region fallback chain building, retryability, sticky behavior, and exhaustion.
sdk_v2/cpp/test/internal_api/model_info_test.cc	Added JSON round-trip coverage for `ModelInfo.detected_region`.
sdk_v2/cpp/test/internal_api/download_test.cc	Updated registry client tests for status-aware HTTP + fallback; added download region-resolution tests.
sdk_v2/cpp/test/internal_api/catalog_cache_test.cc	Added cache save/load coverage for `detected_region`.
sdk_v2/cpp/test/internal_api/c_api_test.cc	Relaxed catalog population assertion to avoid network-dependent expectations.
sdk_v2/cpp/test/internal_api/azure_catalog_test.cc	Expanded live catalog client tests to cover region detection, URL rewriting, stamping, and fallback behavior.
sdk_v2/cpp/test/CMakeLists.txt	Updated unit test sources: added `region_fallback_test`, removed `static_catalog_test`, always includes `azure_catalog_test`.
sdk_v2/cpp/src/util/region_fallback.h	New public/internal utility interface for region-aware retry/fallback execution.
sdk_v2/cpp/src/util/region_fallback.cc	New implementation with proximal-region mapping, retryable classification, sticky region tracking, and diagnostics.
sdk_v2/cpp/src/model_info.h	Added `detected_region` field to model metadata.
sdk_v2/cpp/src/model_info.cc	Implemented JSON serialization/deserialization for `detected_region` (omitted when empty).
sdk_v2/cpp/src/manager.cc	Defaulted catalog region to `auto` and plumbed `DisableRegionFallback` into download + catalog components.
sdk_v2/cpp/src/http/http_client.h	Introduced `HttpResponse` and non-throwing `Http*WithResponse` APIs (status + headers + body).
sdk_v2/cpp/src/http/http_client.cc	Implemented response-aware GET/POST (including header capture + transport failure as `status==0`).
sdk_v2/cpp/src/download/model_registry_client.h	Updated registry client API to accept per-call region and use response-aware HTTP + fallback.
sdk_v2/cpp/src/download/model_registry_client.cc	Implemented region-fallback-driven registry resolution and parsing of response bodies.
sdk_v2/cpp/src/download/download_manager.h	Added config-region tracking and `disable_region_fallback` plumbing.
sdk_v2/cpp/src/download/download_manager.cc	Resolved registry region via explicit config → detected region → eastus; wired fallback-enabled registry client.
sdk_v2/cpp/src/catalog/static_catalog_client.cc	Removed embedded snapshot-backed catalog implementation.
sdk_v2/cpp/src/catalog/live_catalog_client_stub.cc	Removed stub that previously threw when live client sources were absent.
sdk_v2/cpp/src/catalog/catalog_snapshot_data.h	Removed snapshot data header (embedded bytes no longer part of build).
sdk_v2/cpp/src/catalog/catalog_client.h	Updated contract/docs: `MakeCatalogClient` now always constructs live Azure client and accepts region/fallback controls.
sdk_v2/cpp/src/catalog/catalog_client.cc	Retained cached-model + BYO synthesis helper; removed static-vs-live dispatch logic.
sdk_v2/cpp/src/catalog/azure_model_catalog.h	Default catalog URL is now always Azure; added region + fallback configuration fields.
sdk_v2/cpp/src/catalog/azure_model_catalog.cc	Plumbed region/fallback into `MakeCatalogClient` and persisted fetched model infos (incl. region) to cache.
sdk_v2/cpp/src/catalog/azure_catalog_client.h	New live Azure catalog client interface with region detection and fallback-aware fetching.
sdk_v2/cpp/src/catalog/azure_catalog_client.cc	New live Azure catalog implementation: request shaping, region detection, URL rewriting, filter-set atomicity, and fallback.
sdk_v2/cpp/CMakeLists.txt	Removed tools option + static snapshot wiring; always builds live catalog + region fallback sources.

…parsing

…o baijumeswani/catalog

skottmckay · 2026-06-16T02:10:49Z

+constexpr const char* kDefaultRegion = "eastus";
+
+// The catalog and registry gateways reject requests without this User-Agent (HTTP 400).
+constexpr const char* kUserAgent = "AzureAiStudio";


Is now a good time to try a different value here to differentiate traffic?

Unfortunately, no. If we change this to anything other than AzureAiStudio, we receive HTTP 400 response. The server rejects our requests. This is probably by design since they don't requests from random User-Agents.

Will work with the catalog team and update this in a separate commit.

skottmckay · 2026-06-16T02:11:44Z

+constexpr const char* kUserAgent = "AzureAiStudio";
+
+/// Strip leading/trailing whitespace.
+std::string Trim(const std::string& s) {


Should we put all string utils in a single file under src/util? Could move to_lower from src/utils.h and maybe MakeString from src/util/make_string.h into that as well.

skottmckay · 2026-06-16T02:19:29Z

+      const std::string url = regional && pinned_region
+                                  ? BuildRegionalUrl(url_prefix_, url_suffix_, *pinned_region)
+                                  : BuildRequestUrl(base_url_, regional, region_, url_prefix_, url_suffix_);
+      FL_THROW(FOUNDRY_LOCAL_ERROR_INTERNAL,


Should we have a different status for this? I would typically use 'internal' for some implementation logic failure (i.e. our code is broken or did something unexpected), and I wonder if this failure is more likely to be network related.

skottmckay · 2026-06-16T02:26:17Z

-                                 ILogger& logger)
+                                 ILogger& logger, bool disable_region_fallback)
    : cache_directory_(std::move(cache_directory)),
+      config_region_(NormalizeConfiguredRegion(catalog_region)),


nit: If we're not std::move'ing catalog_region now should we take it by reference or maybe as a std::string_view given we're calling to_lower on it?

skottmckay · 2026-06-16T02:42:05Z

Should we have 'real' tests in test/sdk_api for any of the new changes?

Copilot AI review requested due to automatic review settings June 9, 2026 07:06

Copilot started reviewing on behalf of baijumeswani June 9, 2026 07:06 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread sdk_v2/cpp/src/download/model_registry_client.cc Outdated

Comment thread sdk_v2/cpp/src/download/model_registry_client.cc Outdated

Comment thread sdk_v2/cpp/src/catalog/azure_catalog_client.cc Outdated

Comment thread sdk_v2/cpp/src/util/region_fallback.cc Outdated

vercel Bot deployed to Preview June 11, 2026 05:38 View deployment

baijumeswani added 3 commits June 11, 2026 21:20

Enable live catalog with region-aware fallback engine

3a6155b

Add AzureAiStudio User-Agent

cc1e22f

Address pr review comments

a827ec8

baijumeswani force-pushed the baijumeswani/catalog branch from 636f28d to a827ec8 Compare June 12, 2026 04:21

vercel Bot deployed to Preview June 12, 2026 04:21 View deployment

bmehta001 reviewed Jun 13, 2026

View reviewed changes

Comment thread sdk_v2/cpp/src/manager.cc Outdated

bmehta001 reviewed Jun 13, 2026

View reviewed changes

Comment thread sdk_v2/cpp/src/http/http_client.cc Outdated

bmehta001 reviewed Jun 13, 2026

View reviewed changes

Comment thread sdk_v2/cpp/src/http/http_client.h

bmehta001 reviewed Jun 15, 2026

View reviewed changes

Comment thread sdk_v2/cpp/src/download/download_manager.cc Outdated

baijumeswani added 2 commits June 15, 2026 15:27

Default registry region to centralus, httprequestoptions, and config …

fd865f3

…parsing

Merge branch 'main' of https://github.com/microsoft/Foundry-Local int…

c872980

…o baijumeswani/catalog

vercel Bot deployed to Preview June 15, 2026 22:28 View deployment

skottmckay reviewed Jun 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable live catalog with region-aware fallback engine#789

Enable live catalog with region-aware fallback engine#789
baijumeswani wants to merge 5 commits into
mainfrom
baijumeswani/catalog

baijumeswani commented Jun 9, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skottmckay Jun 16, 2026

Uh oh!

baijumeswani Jun 16, 2026

Uh oh!

skottmckay Jun 16, 2026

Uh oh!

skottmckay Jun 16, 2026

Uh oh!

skottmckay Jun 16, 2026

Uh oh!

skottmckay Jun 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

baijumeswani commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How it works

Uh oh!

vercel Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skottmckay Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

baijumeswani Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

skottmckay Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

skottmckay Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

skottmckay Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

skottmckay Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

baijumeswani commented Jun 9, 2026 •

edited

Loading

vercel Bot commented Jun 9, 2026 •

edited

Loading