Proposal: Extract shared sections from `switchyard-server` to make future integratons easier

[grahamking]

Our first integration is with NeMo-Relay. It needs a lot of things that switchyard-server already has, so conceptually we want to do this:

HTTP JSON ──> switchyard-server ──┐
                                  ├──> switchyard-runner ──> libsy + provider clients
Relay JSON ─> NeMo plugin ────────┘

Both adapters decode into switchyard_protocol::Request, invoke the same runner, and encode the returned Response.

This plan would introduce new crate switchyard-runner with these key types:

  AlgorithmSpec  →  Route  →  Runner
  (what to build)   (run it)   (named table, server only) 

The server code would then look like this:

  // Server: TOML → Routes → table                         
  let runner = Runner::load(path)?;                                                                                            
  let route = runner.route(&model)?;                                                                                           
  route.check_caller_format(wire_format)?;
  route.execute(request, Some(observer)).await?;

The plugin would look like this:

  let algorithm = AlgorithmSpec::build(...from plugin config...);
  let route = Route::new(algorithm, clients);
  route.execute(request, Some(observer)).await?;

At setup it reads the Relay JSON config, builds one Route and hold its on SwitchyardRuntime. At intercept time it translates Relay request to switchyard-translation request and calls route.execute.

The full plan follows.

---

Plan: Extract switchyard-runner as shared routing code

Context

The NeMo-Relay plugin mentioned in this plan is in a different branch. Those files will not
be visible right now. Our goal here is to extract switchyard-runner from switchyard-server
to make the NeMo-Relay plugin much simpler when we work on that.

Why

switchyard-server and the NeMo-Relay plugin are two serving surfaces over one routing engine.
Today they each own a private copy of the configured-routing layer.

The duplication is concentrated in one place — turning a declarative algorithm description into an
Arc<dyn Algorithm>, and then driving it. switchyard-server/src/config.rs:999-1271 and
switchyard-nemo-relay-plugin/src/config.rs:361-479 are the same function written twice against two
schemas. Everything around that (HTTP framing, relay marks, statistics, fallbacks) is genuinely
surface-specific and should stay where it is.

The goal is therefore not to refactor switchyard-server. It is to create a shared crate that both
surfaces execute through, so each becomes a thin adapter:

HTTP JSON      -> switchyard-server: axum, headers, decode  -\
                                                              >-> switchyard_protocol::Request -> switchyard-runner
Relay JSON     -> plugin: relay hooks, marks, decode        -/

Goal

Create switchyard-runner owning the schema-neutral algorithm builder, the configured route
execution unit, and the server's TOML deployment loader. Preserve switchyard-server behavior and
public entry points.

Branch scope

This work lands on a branch that does not contain
crates/switchyard-nemo-relay-plugin. The plugin cannot be compiled or tested here.

That has one consequence the implementer must respect: the plugin's requirements are stated below as
hard design constraints, not as aspirations to be settled later. They are cheap to satisfy now
and expensive to retrofit. Section "Plugin fitness constraints" is normative, and section
"Plugin adoption sketch" records what the follow-up branch will do so a reviewer can judge whether
the API actually fits.

Success criteria

Sharing (the point of the work)

• AlgorithmSpec is public, Deserialize, and has no concept of route ids, model capabilities,
  [llm_clients], base URLs, or credentials.
• AlgorithmSpec resolves configured target names through a caller-supplied name-to-ModelId
  map, so one caller can map names to provider model IDs and another can map names to themselves.
• AlgorithmSpec variant fields are public, so a caller can build a spec programmatically instead
  of only through serde.
• Route executes without a route table, a route model ID, or a TOML file. A unit test proves this
  by constructing and executing a single Route the way the plugin will.
• switchyard-server's copy of algorithm construction is deleted, not duplicated.

Preservation

• switchyard-runner has no dependency on Axum, axum-server, Prometheus, OpenTelemetry, TLS,
  nemo-relay-plugin, or server lifecycle code.
• switchyard-server stores a Runner instead of its own route map, configured clients, and
  retained deployment config.
• The OpenAI Chat, OpenAI Responses, Anthropic Messages, decision, token-counting, and model-list
  endpoints preserve their current behavior.
• Existing public switchyard-server APIs continue to compile: ServerState::new,
  ServerState::models, ServerState::caller_auth_kind, ServerState::with_routing_log,
  build_switchyard_router, BoundServer, and config::load_server_state. switchyard-py
  (crates/switchyard-py/src/server_bindings.rs) uses several of these and must not change.
• Existing server tests pass without weakening assertions, except for the one sanctioned message
  change in "Splitting RouteConfig".
• Route::execute accepts the existing RunObserver and forwards it unchanged, so callers retain
  routing-call, answer-call, latency, usage, and routing-overhead visibility.
• Streaming responses remain lazy. The runner does not buffer, poll, spawn, or proxy the returned
  stream.

Non-goals

• Do not change the version-1 deployment TOML schema or its schema version.
• Do not change when environment variables are resolved, or add a secret-resolver abstraction.
• Do not add fallback, retry, routing, or target behavior.
• Do not move request/response translation, HTTP metadata parsing, SSE framing, Prometheus,
  statistics, usage accounting, routing logs, or request spans into the runner.
• Do not share client/backend construction in this change. See "Deliberately not shared".
• Do not reorganize unrelated switchyard-server modules or refactor switchyard-llm-client::run.

Crate layout

crates/switchyard-runner/
├── Cargo.toml
└── src/
    ├── lib.rs        # module declarations and public re-exports
    ├── algorithm.rs  # AlgorithmSpec: schema-neutral algorithm description and builder
    ├── route.rs      # Route, RunOutput, ModelCapabilities, CallerAuthKind, RunnerError
    ├── runner.rs     # Runner: named route table, model metadata, decision description
    └── config.rs     # version-1 TOML deployment loader (moved from switchyard-server)

algorithm.rs and route.rs are the shared layer. config.rs and runner.rs serve the server
today; the plugin uses algorithm.rs and route.rs only.

Dependencies

switchyard-libsy, switchyard-llm-client, switchyard-protocol, serde, serde_json, toml,
reqwest (for the existing validated URL type), tracing. Test-only additions as needed.

Tokio is not a direct production dependency merely because public methods are async. Verify with
cargo tree -p switchyard-runner that server-only dependencies are absent.

Layer 1 — algorithm.rs

This is the shared code that motivates the change.

/// A routing algorithm described by configured target *names*, independent of any
/// deployment schema, transport, or client construction.
#[derive(Clone, Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
pub enum AlgorithmSpec {
    Noop {},
    Random { /* targets, weights, seed */ },
    Passthrough { /* target, subagent_classifier */ },
    LlmClassifier { /* classifier_target, mode, strong/weak, thresholds, escalation, custom */ },
    StageRouter { /* capable_target, efficient_target, picker, threshold, prompts, classifier */ },
    Advisor { /* executor_target, advisor_target, gate settings */ },
}

impl AlgorithmSpec {
    /// Completion destinations in algorithm order. Judge-only targets are excluded.
    pub fn routing_target_names(&self) -> Vec<&str>;

    /// Every target the algorithm may call, including judge-only targets.
    pub fn callable_target_names(&self) -> Vec<&str>;

    /// Builds the algorithm, resolving each configured target name through `targets`
    /// to the `ModelId` the caller's `ClientRouter` is keyed by.
    ///
    /// `context` labels the caller's unit of configuration (a route name for the
    /// server) and appears in error messages. A name absent from `targets` is an
    /// `AlgorithmConfigError`.
    pub fn build(
        &self,
        context: &str,
        targets: &BTreeMap<String, ModelId>,
    ) -> Result<Arc<dyn Algorithm>, AlgorithmConfigError>;
}

The targets map is what makes this shareable. switchyard-server passes the map it already
builds in ServerConfig::build_targets — target name to TargetConfig::id. The plugin passes an
identity map over its own target names, one line at construction. Neither keying policy leaks into
the shared crate, and the "unknown target" error is produced uniformly inside the builder rather
than by each caller.

Move these into algorithm.rs verbatim from switchyard-server/src/config.rs: the algorithm half of
RouteConfig (lines 418-560), routing_target_names / callable_target_names /
classifier_mode (lines 624-901), ClassifierPolicyConfig, ClassifierMode,
LlmClassifierModeConfig, CapabilityClassifierRouteConfig, EscalationClassifierRouteConfig,
CustomClassifierRouteConfig, AdvisorTriggerConfig, StageClassifierConfig,
reject_custom_fields, classifier_field_error, required_classifier_field, build_algorithm
(lines 999-1271), classifier_contract, default_classifier_max_output_tokens, tier_prompts,
default_max_reviews, default_advisor_max_tokens, default_transcript_max_chars,
default_fail_open, and the target-resolution helpers resolve_targets and
resolve_target_model_id (lines 1316-1337). Because build keeps today's &BTreeMap<String, ModelId> parameter, build_algorithm's body moves unchanged apart from matching on &self instead
of &RouteConfig.

AlgorithmConfigError reproduces the server's current message formats verbatim, parameterized by
context — for example "route {context} references unknown target {name}",
"random route {context}: {error}", "llm_classifier route {context} requires {field}". The
server passes the route name, so its configuration error text is unchanged.

The stage_router capable_first experimental warning moves with the builder.

Splitting RouteConfig

The server's [routes.*] table is one flat, internally tagged table that mixes route presentation
(id, context_window, tool_calling, reasoning) with algorithm fields. The schema must not
change, so the server keeps a RouteConfig that holds the four presentation fields plus an
AlgorithmSpec.

Do not use #[serde(flatten)] for this. flatten silently disables deny_unknown_fields, and
the server's configuration tests depend on unknown keys being rejected. Instead, implement
Deserialize for RouteConfig manually:

1. deserialize the route table into a toml::Table (or serde_json::Map);
2. remove and deserialize the four presentation keys;
3. deserialize the remainder — still carrying type — into AlgorithmSpec.

AlgorithmSpec's per-variant deny_unknown_fields then rejects unknown keys as it does today.

One sanctioned message change: serde's unknown-field error lists the expected fields, and that
list no longer contains id, context_window, tool_calling, or reasoning. Update the affected
assertions to keep asserting rejection and the offending key name; do not weaken them to "is an
error". This is the only external message change allowed by this plan.

Required tests in algorithm.rs:

• an unknown key inside a route table is still rejected;
• id, context_window, tool_calling, and reasoning are accepted and are not treated as
  algorithm fields;
• build rejects a target name absent from the targets map, with the current message;
• an identity map (name to name) produces a working algorithm — the plugin's case.

Layer 2 — route.rs

One configured algorithm plus the clients its targets resolve to. This is the shared execution unit
and the type the plugin holds directly.

/// A configured algorithm and the per-target clients its calls resolve through.
/// Immutable after construction; cheap to clone.
#[derive(Clone)]
pub struct Route { /* Arc<RouteInner> */ }

pub struct RunOutput {
    pub selected_model: ModelId,
    pub response: Response,
}

impl Route {
    pub fn new(algorithm: Arc<dyn Algorithm>, clients: ClientRouter) -> Self;

    pub fn with_caller_auth(self, kind: Option<CallerAuthKind>) -> Self;
    pub fn with_capabilities(self, capabilities: ModelCapabilities) -> Self;
    pub fn with_count_tokens_target(self, target: Option<CountTokensTarget>) -> Self;

    pub fn algorithm_name(&self) -> &str;
    pub fn capabilities(&self) -> ModelCapabilities;
    pub fn caller_auth(&self) -> Option<CallerAuthKind>;

    /// Rejects a caller wire format incompatible with this route's forwarded credentials.
    pub fn check_caller_format(&self, input_format: WireFormat) -> Result<(), RunnerError>;

    pub async fn execute(
        &self,
        request: Request,
        observer: Option<RunObserver>,
    ) -> Result<RunOutput, RunnerError>;

    /// Completes routing-time calls and returns the outcome without serving its answer target.
    pub async fn decide(&self, request: Request) -> Result<RoutingOutcome, RunnerError>;

    pub async fn count_tokens(&self, request: Request) -> Result<Value, RunnerError>;
}

Route::new takes only what execution needs; the optional server metadata is set through builder
methods so the plugin ignores all of it.

check_caller_format is a separate method rather than an input_format parameter on execute.
That mirrors today's structure: the check lives in the server's single resolve_route helper
(crates/switchyard-server/src/lib.rs:890-905) and therefore already covers the completion,
decision, and count_tokens endpoints. Folding it into execute would silently drop it from
count_tokens. The server keeps resolve_route, which becomes route lookup plus
check_caller_format.

Move from switchyard-server/src/lib.rs: RouteEntry (becomes RouteInner), ModelCapabilities,
CallerAuthKind, CountTokensTarget, run_decision_only, and serve_decision_dependency.

ModelCapabilities stays a small Clone + Copy + Default value with public fields, since the
server serializes them. CallerAuthKind becomes public — RunnerError carries it, and
CallerAuthKind::as_str backs the public ServerState::caller_auth_kind.

Streaming contract

Route::execute returns the Response produced by switchyard_llm_client::run without consuming
it. For LlmResponse::Stream: do not poll it, proxy it through a channel, spawn a task, aggregate
it, alter event order or terminal behavior, or intercept errors the caller hits while polling.
Dropping the stream must drop the upstream provider response, preserving cancellation and
backpressure. For LlmResponse::Agg, return the aggregate unchanged.

Observability contract

Route::execute passes the supplied RunObserver unchanged to switchyard_llm_client::run and
must not buffer, reorder, rename, or translate observations. Callers keep receiving
RunObservation::LlmCall, RunObservation::AnswerCall, and RunObservation::RoutingOverhead with
their existing normalized fields.

Observers stay request-scoped values owned by callers. Do not add locks or an observer registry to
Route or Runner. The runner initializes no metrics registry and no tracing subscriber.

Request and wire-format invariant

The runner accepts a normalized switchyard_protocol::Request and never decodes raw JSON.
input_format describes the caller protocol and is used only for forwarded-auth validation. It must
not select the provider-facing format; the configured target client stays authoritative.

Preserve current Request.metadata.wire_format behavior: the server must not begin populating it,
the runner must not populate or rewrite it, and a caller that deliberately supplies a provider-format
hint retains existing TranslatingLlmClient behavior. (The plugin does set it deliberately today,
in both directions — runtime.rs:77 and client.rs:77 — which this rule leaves untouched.)

Layer 3 — runner.rs

The named route table. Server-shaped; the plugin never constructs one.

#[derive(Clone)]
pub struct Runner { /* Arc<BTreeMap<ModelId, Route>>, Option<Arc<DeploymentConfig>> */ }

impl Runner {
    pub fn load(path: impl AsRef<Path>) -> Result<Self, RunnerError>;
    pub fn new(routes: impl IntoIterator<Item = (ModelId, Route)>) -> Result<Self, RunnerError>;

    /// A cloned handle, so callers hold no borrow across an await.
    pub fn route(&self, model: &str) -> Option<Route>;

    pub fn models(&self) -> impl Iterator<Item = ModelInfo<'_>>;
    pub fn caller_auth_kind(&self, model: &str) -> Result<Option<&'static str>, RunnerError>;

    /// Resolves a routing outcome through the configured target names of `model`'s route.
    pub fn describe_decision(
        &self,
        model: &ModelId,
        outcome: &RoutingOutcome,
    ) -> Option<DecisionDescription>;
}

pub struct ModelInfo<'a> {
    pub id: &'a ModelId,
    pub algorithm: &'a str,
    pub capabilities: ModelCapabilities,
}

pub struct DecisionDescription {
    pub selected: DecisionTarget,
    pub fallbacks: Vec<DecisionTarget>,
}

pub struct DecisionTarget {
    pub target: String,
    pub model: ModelId,
    pub format: WireFormat,
    pub base_url: String,
    pub extra_body: BTreeMap<String, Value>,
}

route() returning an owned handle removes the lifetime problem that otherwise forces a borrow
across .await.

describe_decision replaces ServerState::decision_response
(crates/switchyard-server/src/lib.rs:335-368). It returns owned values because the decision
endpoint is low-volume and public lifetime parameters tied to Runner internals are not worth it.
Preserve the existing selected/fallback ordering, the deterministic first-match behavior when
several configured targets share one provider model ID, and the all-or-nothing None when any model
has no callable target configuration. Response translation stays in the server: the server re-nests
format/base_url under llm_client so the endpoint's JSON shape is byte-identical.

Runner::new keeps the current validation from ServerState::new_with_capabilities: trim route
model IDs, reject empty IDs, reject duplicates, and reject an empty route set.

Configuration extraction

Move crates/switchyard-server/src/config.rs into crates/switchyard-runner/src/config.rs:

• keep the TOML schema, deny_unknown_fields, and schema version unchanged;
• keep URL, retry, header, credential, target-reference, and algorithm validation unchanged;
• keep environment lookup timing and errors unchanged;
• keep build_backend, build_clients, build_route_clients, build_count_tokens_target, and
  count_tokens_priority unchanged;
• delegate algorithm construction to AlgorithmSpec::build, resolving target names to
  TargetConfig::id;
• make the loader construct Runner instead of ServerState;
• move the existing inline configuration tests with the file.

Runner::load owns the path-context wrapping that load_server_state performs today
(crates/switchyard-server/src/config.rs:34-45) — both failed to read server config {path}: … and
invalid server config {path}: …. The server's compatibility wrapper must not re-wrap, or the path
appears twice.

DeploymentConfig and the serde configuration structs stay private. Only AlgorithmSpec, Route,
Runner, their output and metadata types, and RunnerError form the supported API.

Error model

Two public error types, both implementing Display and Error with typed sources preserved. Do not
add HTTP status codes or public response text to either, and do not add a dependency solely for
error derives.

AlgorithmConfigError — configured target absent from the targets map; missing, conflicting, or invalid
algorithm settings; libsy algorithm construction failure.

RunnerError — must distinguish at least:

• configuration or configuration-file failure (wrapping AlgorithmConfigError where applicable);
• missing request model;
• unknown route model;
• incompatible caller wire format for forwarded credentials, carrying the CallerAuthKind so
  the server can rebuild its current message naming the provider and expected endpoint;
• unsupported token counting for a route;
• a routing outcome containing a model with no callable target configuration — today's 500 at
  crates/switchyard-server/src/lib.rs:706-711;
• algorithm execution failure retaining LibsyError;
• direct client failure retaining LlmClientError.

The server stays responsible for converting these into its current status codes and
wire-format-specific error bodies. External error messages must not change.

Server integration

#[derive(Clone)]
pub struct ServerState {
    runner: Runner,
    metrics: prometheus::Registry,
    stats: StatsAccumulator,
    routing_log: Option<SharedRoutingLog>,
    track_cache_eligibility: bool,
}

Add pub fn from_runner(runner: Runner) -> ServerResult<Self>, which initializes the metrics
registry and StatsAccumulator from Runner::models. Keep the public tuple-based
ServerState::new by building Routes and a Runner, then delegating. Keep ServerState::models
and ServerState::caller_auth_kind as delegations.

Keep switchyard_server::config::load_server_state at its current path as a thin wrapper:

pub fn load_server_state(path: impl AsRef<Path>) -> ServerResult<ServerState> {
    ServerState::from_runner(Runner::load(path).map_err(ServerError::from)?)
}

resolve_route keeps its current shape and responsibilities — decode the body, require a non-empty
model, look the route up through Runner::route, call Route::check_caller_format, build the
Request — and returns an owned Route instead of a &RouteEntry. All three endpoint families
keep calling it, so validation order and status codes are unchanged.

• Completion endpoints: replace switchyard_llm_client::run with Route::execute. The server
  still computes the cache probe from the raw body first, builds its statistics/routing-log
  observer, derives served_model from RunOutput, accounts usage through usage_metrics::observe,
  encodes the response, and attaches routing headers.
• Decision endpoint: replace run_decision_only with Route::decide and decision_response
  with Runner::describe_decision. Response encoding and endpoint JSON serialization stay in the
  server.
• Token counting: replace direct CountTokensTarget access with Route::count_tokens. The
  server keeps Anthropic-specific HTTP errors and response serialization.
• Models, dry-run, startup banner: read Runner::models. Ordering is preserved by the
  BTreeMap<ModelId, _> storage. Do not change any output formatting.

Plugin fitness constraints

Normative. Each is cheap now and expensive later. Each is verifiable on this branch.

1. Route must be constructible and executable without a Runner, a route model ID, or a TOML
   file. The plugin dispatches on inbound protocol, and its caller-supplied model is an arbitrary
   upstream model name — it has no route table to look that up in.
2. Route::execute must not read request.llm_request.model to select anything. Route selection is
   the caller's job.
3. AlgorithmSpec::build must accept an identity targets map (name -> ModelId::from(name)) and
   must not assume the resolved ModelId differs from the configured name. The plugin keys its
   ClientRouter by semantic target name and rewrites to the provider model inside its own client
   (crates/switchyard-nemo-relay-plugin/src/client.rs:101-103).
4. AlgorithmSpec variant fields must be public. The plugin derives Random targets and weights
   from per-target weight fields in its own schema rather than from a targets/weights pair, so
   it must be able to construct the spec programmatically.
5. Route must accept any Arc<dyn RoutedLlmClient> through ClientRouter. The plugin's
   TargetClient is not a TranslatingLlmClient.
6. Nothing on the shared path may require ModelCapabilities, CallerAuthKind,
   CountTokensTarget, or a config name. All are optional server metadata.

Guard rail: route.rs carries a test that builds a Route exactly the way the plugin will — an
identity targets map, a stub RoutedLlmClient keyed by semantic name, no Runner, no route model
ID, no TOML — and executes a request through it. If that test needs a workaround, the API is wrong.

Deliberately not shared

Stated so a reviewer can see these were considered rather than missed.

Client and backend construction. build_backend plus [llm_clients]
(crates/switchyard-server/src/config.rs:942-993) and the plugin's TargetBinding::prepare /
TargetClient implement different policies: one TranslatingLlmClient shared across targets versus
one per target, api_key_env versus header_env, provider-model-ID keying versus semantic-name
keying, and the plugin's drop_caller_extra_body and inbound-header stripping. Sharing these would
force a change to how the server's decision endpoint resolves target names. Left as a follow-up:
adopting the plugin's semantic-name keying in the server would also delete the ambiguous
target/model warning at crates/switchyard-server/src/config.rs:92-98 and the first-match tie-break
in describe_decision.

Translation. Already shared through switchyard-translation. The server uses the default-policy
helpers (decode_request, encode_aggregated_response, encode_stream); the plugin uses
TranslationEngine with a stricter policy that rejects lossy conversions and non-Info diagnostics
(crates/switchyard-nemo-relay-plugin/src/translation.rs:48-82). That is a policy difference, not
duplicated logic.

Fallback, marks, and stream-commitment semantics. Plugin-only. The server has no counterpart and
should not grow one.

Implementation sequence

1. Start

Remember that the plugin crate is absent on this branch.

2. Add the crate and the shared algorithm builder

• Add crates/switchyard-runner to the workspace with the minimal manifest.
• Add lib.rs and algorithm.rs; move the algorithm half of RouteConfig and build_algorithm.
• Implement the manual RouteConfig split described above, still inside switchyard-server, and
  point it at AlgorithmSpec::build.

cargo test -p switchyard-runner -p switchyard-server
cargo clippy -p switchyard-runner --all-targets -- -D warnings

Success: the server's configuration tests pass with the algorithm builder deleted from
switchyard-server, and the only assertion churn is the sanctioned unknown-field message.

3. Add Route

• Add route.rs with Route, RunOutput, RunnerError, ModelCapabilities, CallerAuthKind,
  and CountTokensTarget.
• Move run_decision_only and serve_decision_dependency.
• Add the plugin-shaped guard-rail test.

cargo test -p switchyard-runner

4. Add Runner and move the deployment loader

• Add runner.rs; move config.rs and its inline tests; make the loader build a Runner.
• Reduce switchyard-server/src/config.rs to the load_server_state wrapper without changing its
  public path.
• Confirm configuration errors keep their contextual path and message, and that the path is not
  wrapped twice.

5. Delegate the server

• Add Runner to ServerState; add from_runner; preserve ServerState::new.
• Rework resolve_route to return an owned Route.
• Replace completion execution, decision, token counting, model listing, dry-run, and startup output
  with delegations.
• Map RunnerError through the existing HTTP error helpers.
• Remove the duplicated route state from ServerState only after every call site uses the runner.

cargo test -p switchyard-server

Pay particular attention to existing tests for retries, candidate fallback, streaming, provider
errors, request metadata, usage, statistics, and routing logs.

6. Remove residue

• Remove imports, fields, helpers, and tests made redundant by the move.
• Do not clean up unrelated server comments, formatting, or dead code.

rg 'switchyard_server|axum|prometheus|opentelemetry|routing_log|nemo_relay' crates/switchyard-runner
cargo tree -p switchyard-runner
git diff --find-renames --stat origin/main...HEAD

7. Full validation

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
uv run ruff check .
uv run pytest tests/

The Python checks should be unaffected, but repository policy requires them before a commit. Report
any command not run and why.

Test ownership

switchyard-runner owns tests for: AlgorithmSpec deserialization and the presentation/algorithm
split; unknown-field rejection; algorithm construction under both provider-ID and identity target
maps; AlgorithmConfigError variants; TOML parsing, validation, and client construction;
route lookup and RunnerError variants; caller-format compatibility; observer forwarding;
decision-only execution; token-count target selection; lazy propagation of streaming responses
without polling them; and the plugin-shaped single-Route guard rail.

switchyard-server retains tests for: endpoint paths and methods; JSON and header extraction;
request and response translation; SSE framing and terminal events; HTTP status and error mapping;
selected-model headers; model-list JSON shape; decision endpoint JSON shape; stats, metrics, usage
accounting, and routing logs; TLS and lifecycle; and the compatibility wrappers.

Move tests when they primarily exercise runner behavior. Do not copy the same behavior test into
both crates. Existing end-to-end server tests may keep covering runner behavior transitively.

Plugin adoption sketch

Non-normative; records what the follow-up branch does, so this API can be reviewed for fitness.

// Once, at register(). The plugin's ClientRouter is keyed by semantic target name,
// so its target map is the identity over its configured names.
let by_name: BTreeMap<String, ModelId> =
    targets.keys().map(|name| (name.clone(), ModelId::from(name.as_str()))).collect();
let algorithm = spec.build("switchyard", &by_name)?;
let clients = ClientRouter::new(
    targets.iter().map(|(name, t)| (ModelId::from(name.as_str()), t.client.clone())).collect(),
);
let route = Route::new(algorithm, clients);

// Per request, replacing SwitchyardRuntime::drive:
let RunOutput { response, .. } = route.execute(request, Some(observer)).await?;

The plugin deletes its AlgorithmConfig, LlmClassifierAlgorithmConfig, StageFallbackConfig,
build_algorithm, and per-request ClientRouter construction — roughly 200 lines — and inherits
passthrough, advisor, and custom-classifier support it does not have today. Its
session_affinity field becomes classify_trigger, which fixes the current compile break. Its
Random handling keeps deriving targets and weights from per-target weight fields, constructing
AlgorithmSpec::Random directly rather than through serde — which is why constraint 4 exists.

Its per-target validation, TargetClient, translation policy, fallback-to-default-target, routing
marks, and stream-commitment logic all stay in the plugin.

Review checklist

• No production unwrap, expect, or panic was introduced.
• No lock is held across .await; Route and Runner are immutable after construction.
• No task or channel was added to the streaming path; LlmResponse::Stream stays lazy and preserves
  event order, partial chunks, errors, and cancellation.
• RunObserver delivery order and event classification are unchanged.
• Provider retry and candidate-fallback behavior still comes from switchyard-llm-client.
• Request metadata and target wire-format resolution are unchanged.
• Server error statuses, codes, and redaction are unchanged.
• TOML configuration, defaults, validation, and environment behavior are unchanged, and unknown
  fields are still rejected.
• Public server APIs are preserved; switchyard-py compiles unchanged.
• switchyard-runner has no server-only dependencies and no reference to the plugin.
• All six plugin fitness constraints hold, and the guard-rail test needs no workaround.
• No unrelated refactor or formatting churn is included.

Expected change shape

Mostly moved code. algorithm.rs and config.rs are near-verbatim moves out of
switchyard-server/src/config.rs; route.rs is a near-verbatim move out of
switchyard-server/src/lib.rs. New code should be roughly 200-300 lines: the AlgorithmSpec public
surface, the manual RouteConfig deserializer, Route's builder methods,
RunnerError, and server delegation.

The final review should show a new crate containing the shared routing layer, substantial deletions
from switchyard-server, small server delegation changes, unchanged external behavior, and no new
execution or transport mechanism.

If the implementation starts requiring generalized hooks, configurable adapters, an event bus, a new
configuration model, or duplicated tests across crates, stop and reduce the design back to the
existing behavior behind AlgorithmSpec and Route.

[grahamking]

Implementation details diverged from the plan to end up with simpler code. Key objects and usage are the same though.