Skip to content

SimpleContext.add() content is silently discarded from generation — add a safety net #1597

Description

@planetf1

Problem

SimpleContext.view_for_generation() always returns [] by design (mellea/stdlib/context/simple.py) — it's a stateless context and never forwards history to the model. But .add() still exists and returns a chainable context, which reads like it records the turn for the next call. A caller who .add()s a message and then separately passes an unrelated (or empty) action to generate_from_context ends up sending a request with no real user content at all — and nothing errors. The model just receives an empty conversation.

Concrete instance

This is exactly what happened in #1587's live Ollama telemetry tests, before it was caught:

ctx = SimpleContext()
ctx = ctx.add(Message(role="user", content="Say 'hello' and nothing else"))
mot, _ = await backend.generate_from_context(
    Message(role="assistant", content=""), ctx, model_options=model_options
)

The .add()ed message never reaches the model — the actual generation input is the empty assistant message. Granite 4.2 rambles for 2000+ tokens on the resulting empty prompt, which was one of two root causes behind sustained CI stalls on that PR (see its "CI stability fixes" section).

Proposed fix

  1. Docs: SimpleContext.add()'s own docstring should state that content added here is not forwarded to generation — today only view_for_generation() mentions the discard, and it's easy to read .add() in isolation and assume otherwise.
  2. Code: add a guard in the shared message-assembly path (each backend's _generate_from_chat_context_standard or equivalent) that raises or logs a warning when the fully-assembled messages list sent to the model has no non-empty user-role content. This is a universal invariant, not SimpleContext-specific, and would have caught this exact bug at test-authoring time instead of during a multi-hour CI stall investigation.

Related

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions