Skip to content

feat: author-pluggable message serialization surface #1565

Description

@AngeloDanducci

Today a component's only serialization extension surface is
format_for_llm() -> TemplateRepresentation | str (mellea/core/base.py:553).
The formatter and each backend then decide how the TemplateRepresentation becomes a
provider wire message. This works for the fixed set of fields Mellea knows about
(role, content, images, audio, thinking, tool_calls, tool_call_id, and now
tool_name), but a component author cannot:

  • emit a field Mellea does not model (e.g. a provider-specific key), or
  • take over serialization entirely and return a raw provider dict for their component.

This surfaced while closing #1030 item 2: we had to add a first-class tool_name field to
TemplateRepresentation/Message (see PR for #1030) precisely because there was no
author-facing channel to declare it otherwise. Each new provider-specific field currently
costs a core-type change plus per-backend wiring. That does not scale.

Why is this separate from #1030 ?

#1030 item 2 ("infrastructure for better message serialization … so we can write custom
serialization when required, ie for tool results") was closed by making the existing
IR-field path (TemplateRepresentation -> Message -> backend) honored consistently
across all backends. This issue is the design follow-up:
a general extension point so authors don't have to petition core for every new field.

Problem

  • New fields require editing TemplateRepresentation (mellea/core/base.py), Message
    (mellea/stdlib/components/chat.py), message_from_template_representation, and every
    backend's inline serialization loop
    (watsonx.py, ollama.py, utils.py/HF, plus message_to_openai_message).
  • There is no way for a component to say "serialize me as exactly this provider dict."
  • Wire shapes already diverge per provider (Ollama's tool-call shape != OpenAI's), so a
    passthrough dict would also need to declare which provider(s) it targets.

Options to evaluate

  1. Extra-fields passthrough on TemplateRepresentation - e.g. an optional
    extra: dict[str, Any] | None (or provider_fields: dict[str, dict] keyed by provider)
    that backends merge into the message dict after the known fields.
  2. Registered serializer - a component (or type) registers a
    Callable[[Component, provider], dict] that a backend consults before falling back to
    the default path. Most flexible; needs a clear provider-identity contract and a
    precedence rule vs. the known fields.
  3. format_for_llm returns a provider message directly - widen the return union to
    allow a raw provider dict tagged with its provider. Simplest for the author, but leaks
    provider shape into component code and needs a guard for provider mismatch.

Each needs: a provider-identity contract (how a component targets Ollama vs.
OpenAI-compatible vs. HF), a precedence/merge rule against Mellea's known fields, and a
validation story (bad/unknown keys).

Acceptance criteria

  • A component author can emit a provider field Mellea does not model, without a core-type
    change, on at least the OpenAI-compatible + Ollama paths.
  • Precedence between author-supplied fields and Mellea's known fields is documented and
    tested.
  • Provider mismatch (author targets provider X, request goes to provider Y) is handled
    explicitly (ignored or errored tbd), not silently mis-serialized.
  • Unit tests per affected backend asserting the author-declared field reaches the wire
    message; a doc/example under docs/examples/.

Nice to have

References

Metadata

Metadata

Labels

needs-designProblem is clear; implementation approach needs design discussion before work starts

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions