Skip to content

[FEATURE] Track metadata and cost-aware model routing #3860

Description

@JackYPCOnline

SDK Language

Python initially; evaluate TypeScript parity as each public API is designed.

Problem Statement

PR #3846 adds classifier-driven input-complexity routing while intentionally keeping model capability, quality, and price out of candidate declaration order. Several useful extensions were deferred so the initial strategy stays focused. Without one tracking issue, those extensions could be implemented inconsistently or folded into InputComplexityStrategy as unrelated policy.

Proposed Solution

Track the follow-up work as separate, composable API changes:

  • Add optional, keyword-only typed metadata to RoutingCandidate without breaking existing construction.
  • Model capability/quality and pricing as separate dimensions; do not infer either from candidate order.
  • Design a separate cost-aware routing strategy that consumes caller-provided metadata.
  • Keep authoritative pricing and hidden provider/documentation lookups out of InputComplexityStrategy.
  • Evaluate an explicit opt-in classifier agent or factory for customers who need tool-assisted lookup; never inherit parent-agent tools implicitly.
  • Add customer documentation and examples after the public APIs complete API bar-raising.
  • Evaluate and track the corresponding TypeScript API for cross-SDK parity.

Use Case

Customers should be able to balance request complexity, model quality, and budget without encoding assumptions in list order or coupling cost policy to complexity classification. Deployments with private models should be able to provide their own metadata, while deployments that need external capability lookup should opt into that latency and tool surface explicitly.

Scope and Compatibility

InputComplexityStrategy remains responsible only for model-aware complexity selection and its documented recovery behavior. Runtime failover remains explicit through fallback. Existing RoutingCandidate, ModelRouter, and FallbackStrategy usage must remain compatible.

Additional Context

Metadata

Metadata

Assignees

Labels

area-modelRelated to models or model providersarea-multiagentMulti-agent relatedenhancementNew feature or requestpythonPull requests that update python code

Fields

Language

Python

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions