Skip to content

feat(model-routing): add ModelRouter core and candidate resolution#8

Closed
JackYPCOnline wants to merge 6 commits into
feature/model-routingfrom
feature/model-routing-core
Closed

feat(model-routing): add ModelRouter core and candidate resolution#8
JackYPCOnline wants to merge 6 commits into
feature/model-routingfrom
feature/model-routing-core

Conversation

@JackYPCOnline

Copy link
Copy Markdown
Owner

Description

Phase B of model routing (design team/designs/0016-model-routing.md): the ModelRouter core. Stacked on the Phase A per-call-model seam (base branch feature/model-routing), so this PR's diff is Phase B only.

Adds the strands.models.routing package and teaches Agent to accept a router through model=. Per-call selection middleware lands in a follow-up, so routing is inert here: an agent with a router uses the router's first (default) candidate, and single-model usage is unchanged.

Public API (provisional, pre bar-raising)

from strands.models.routing import ModelRouter, RoutingCandidate

# First candidate is the default exposed as agent.model.
router = ModelRouter(models=[haiku, sonnet], strategy=SomeStrategy())
agent = Agent(model=router)

# Optional names/descriptions for semantic strategies:
ModelRouter(
    models=[
        RoutingCandidate(model=haiku, name="routine", description="Simple tasks."),
        RoutingCandidate(model=opus, name="complex", description="Harder reasoning."),
    ],
    strategy=SomeStrategy(),
)
  • ModelRouter is a reusable, immutable Plugin over an ordered candidate sequence; the same router can back multiple agents.
  • Candidates are Model, a model-id string (resolved to BedrockModel), a nested ModelRouter, or a RoutingCandidate.
  • RoutingStrategy is async select(context) -> RoutingCandidate; RoutingContext carries the request data and normalized candidates.
  • Stateful candidates are rejected recursively at construction; agent.model resolves to the first candidate as a concrete model.

Related Issues

Refs strands-agents#364

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

  • I ran hatch run prepare

18 unit tests cover normalization, first-candidate default, nested resolution, stateful/duplicate/type guards, and Agent integration. Added tests_integ/models/test_model_routing.py (real Bedrock) exercising the Agent -> router -> InvokeModelStage -> concrete model path end to end. Full unit suite passes; ruff + mypy clean.

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my feature works
  • My changes generate no new warnings

Draft, stacked on the Phase A seam (feature/model-routing). Routing is inert until the selection-middleware phase.

@github-actions github-actions Bot added the enhancement New feature or request label Jul 24, 2026
strands-agent and others added 6 commits July 24, 2026 15:31
…nds-agents#3463)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…trands-agents#3462)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: opieter-aws <opieter@amazon.com>
…s#3465)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Add the strands.models.routing package aligned with design 0016:
RoutingCandidate (model plus optional name/description), ModelRouter (a
reusable, immutable Plugin over an ordered candidate sequence), the
RoutingStrategy protocol (select -> RoutingCandidate), and RoutingContext.
The router normalizes model/string/nested-router/RoutingCandidate inputs,
exposes the first candidate resolved to a concrete model as the default,
and recursively rejects stateful candidates.

Agent now accepts model=ModelRouter(...): it exposes the router's default
as agent.model and registers the router as a plugin. Per-call selection
middleware lands in a follow-up, so routing is inert (uses the first
candidate) for now and single-model usage is unchanged.

Includes an end-to-end integration test (real Bedrock) covering the
Agent -> router -> InvokeModelStage -> concrete model path.

Refs strands-agents#364
@JackYPCOnline
JackYPCOnline force-pushed the feature/model-routing-core branch from c78b099 to 8d45480 Compare July 24, 2026 18:43
@JackYPCOnline

Copy link
Copy Markdown
Owner Author

Superseded by the upstream Phase B PR against strands-agents/harness-sdk (rebased onto main after Phase A merged).

@github-actions

Copy link
Copy Markdown

Documentation Preview Failed

The documentation deployment encountered an error. Please check the deployment logs for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants