Skip to content

Output-schema validation: ToolOutput::Ok needs a structured data arm before a contract can check what a tool returned #3285

Description

@macanderson

PR #3281 landed ToolContract with no output_schema, deliberately: there is nothing to validate against. ToolOutput::Ok { content: String } carries prose, so "references, not payloads" (#2694 §4) is unenforceable and a tool's declared output shape would be a field nothing reads.

Input validation already exists (#3144, crates/stella-tools/src/registry/validate.rs). This is the other direction.

What to build

  1. ToolOutput::Ok { content: String, data: Option<Value> } — additive, #[serde(default, skip_serializing_if = "Option::is_none")] so every payload written before the field round-trips byte-identically (invariant sync(stella): Gemini/Vertex/Bedrock providers, custom tools, skills from monorepo #4; ErrorClass in crates/stella-protocol/src/tool.rs is the pattern, including its tests).
  2. ToolContract::output_schema: Option<Value>.
  3. Registry-side validation after execution, in a sibling submodule beside registry/validate.rsregistry.rs is a god file closed to growth.
  4. A failure is ErrorClass::Internal, not InvalidInput: a tool that returns something its own contract forbids is a tool defect, and the class axis exists so that never counts as model misuse. Log and surface it; never pass it silently to the model.

Why it matters beyond tidiness

The schema and the hand-written parser can silently disagree today, in both directions. Validating both ways is what catches a handler drifting from its contract.

Witness

A tool whose declared output_schema contradicts what it returns yields a classified Internal error naming the failing field; a tool with no declared output_schema is untouched.

Refs #2716, #2694, #3144.

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageUntyped request — convert by adding bug / feature / epic

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions