feat: gateway upgrades

[TEJASNARAYANS]

What Changed?

Briefly describe what this PR changes.

Why?

Explain the problem this solves and why this approach was chosen.

How to Test?

List clear steps for reviewers to verify the change.

Release Label

Select one semantic version bump intent for this PR:

• major - breaking change, next release bumps major version
• minor - backward-compatible feature, next release bumps minor version
• fix - backward-compatible bug fix, next release bumps patch version
• No label (defaults to patch release)

If you do not have permission to apply labels, mention the intended release label here and a maintainer will set it.

Checklist

• I have read the CONTRIBUTING.md guide.
• My code follows the project's style guidelines.
• I have added tests that prove my fix is effective or my feature works.
• I have updated documentation where needed.

[greptile-apps]

Greptile Summary

This PR adds per-credential LLM gateway routing overrides across the full stack: each AIProvider and Integration row can now independently specify routing_mode (inherit/gateway/direct), a custom gateway_model, gateway_interface (litellm_shim vs native_openai), per-credential gateway URL, custom auth header/secret, and arbitrary extra headers. The change also introduces the resolve_lm_call helper in lm_resolver.py that propagates credential routing context into GEPA optimization flows.

• Gateway URL double-normalization risk: _sanitize_gateway_base_url pre-normalizes the stored URL at write time; _normalize_base_url normalizes it again at runtime for the effective interface. Changing only gateway_interface on a PUT (without updating gateway_base_url) causes the native_openai \u2192 litellm_shim transition to produce http://host/v1/litellm instead of http://host/litellm, silently breaking gateway routing.
• gateway_model dead parameter: accepted and passed to _validate_routing_and_api_key but never read, so a gateway_model on a direct-routing credential is silently ignored.
• gateway_extra_headers stored as plaintext JSON: unlike gateway_auth_secret which is encrypted at rest, header values (which may include auth tokens) are stored unencrypted.

Confidence Score: 3/5

Safe to merge with caution; the URL double-normalization issue can silently break gateway routing for credentials whose gateway_interface is updated independently of their gateway_base_url.

The gateway_base_url pre-normalization at write time combined with re-normalization at call time is not idempotent across interface changes. Changing gateway_interface from native_openai to litellm_shim without touching the URL produces a malformed api_base that silently yields 404s on every LiteLLM call for that credential.

app/api/v1/routes/aiproviders.py (_sanitize_gateway_base_url and the update route handling) and app/services/ai/llm_gateway.py (_normalize_base_url / normalize_bifrost_url interaction with pre-stored URLs).

Important Files Changed

Filename	Overview
app/services/ai/llm_gateway.py	Major refactor introducing CredentialRoutingContext, per-credential gateway interface selection, custom auth headers/secrets, and extra_headers support. Contains a URL double-normalization bug when gateway_interface changes without updating gateway_base_url.
app/api/v1/routes/aiproviders.py	Adds per-credential routing/gateway fields with validation and URL sanitization. gateway_base_url is pre-normalized at storage time, which is the root of the double-normalization issue.
app/services/optimization/lm_resolver.py	New resolve_lm_call cleanly propagates CredentialRoutingContext; raises RuntimeError instead of silently returning None when no AI provider is found.
app/services/ai/llm_resolver.py	get_llm_provider_and_model expanded to handle credential_id lookups, org-wide default rows, and gateway_model resolution; logic is consistent and correct.
app/models/schemas.py	Well-validated new fields for gateway routing on AIProvider and Integration schemas.
app/models/database.py	New columns for routing_mode, gateway fields, and gateway_extra_headers (JSON) on AIProvider and Integration; aligns with migrations.
app/services/ai/llm_gateway_settings.py	Org-level gateway settings extended with gateway_interface; normalization correctly delegates to normalize_bifrost_native_url or normalize_bifrost_url based on effective interface.
app/services/judge_alignment/gepa_bridge.py	_resolve_api_key replaced by _resolve_lm_call which carries credential context through gateway and model resolution steps.
app/migrations/053_integration_routing_settings.py	Adds routing_mode and gateway_model columns to aiproviders and routing_mode to integrations; idempotent via column-exists checks.
frontend/src/lib/gatewayRouting.ts	New utility module for gateway routing checks; routesViaGateway checks effective_routing==='gateway' which the backend never emits, but the routing_mode fallback provides correct behavior.
app/api/v1/routes/integrations.py	Adds routing_mode to Integration CRUD; _integration_response helper computes effective_routing correctly.
app/services/ai/llm_service.py	Propagates CredentialRoutingContext into apply_llm_gateway and resolve_litellm_model; credential context properly threaded through the call chain.

_{Reviews (5): Last reviewed commit: "fix: some more tests" | Re-trigger Greptile}

[greptile-apps]

N+1 org-settings queries in list endpoints

_scrub_for_response calls get_credential_effective_routing_label → resolve_effective_routing → _get_org_raw_settings, which issues a DB query for each provider in the list. For an org with many AI providers the list endpoint issues N redundant identical queries for the same org row. The same pattern applies to list_integrations. Fetching org settings once before the loop would eliminate the redundant queries.