Skip to content

chat: read_manifest returns a card's whole data block uncapped, so one call can exhaust the context window #708

Description

@Aristocles

What

Every other read tool bounds its output. read_manifest_meta omits data
entirely. read_manifest_rows truncates arrays past 10 rows and replaces nested
long arrays with {omittedArray:true, count:N}, and says so in its description
so the model knows to re-fetch by a deeper path. get_recent_activity returns a
one-line-per-card summary.

read_manifest returns {meta, description, schema, data} with data serialised
in full, no cap and no truncation marker. On a card with a few thousand rows
(the HAE-fed sleep card has been over a thousand) that is one tool result larger
than the whole rest of the prompt.

Why

The tool's own description still steers the model to it: write_manifest_data
tells the model to "first call read_manifest to get the current data", and
patch_manifest says "ALWAYS call read_manifest first". So the model reaches for
it on exactly the cards where it is most expensive, and one such call can push
the turn past the model's context window. The gateway answers 400, which is
currently reported as a transient blip (see the sibling issue), so the
conversation is dead with no explanation.

Why now

#694 tripled the round-trip count and the conversation context budget, which
makes a single oversized tool result more likely to be the thing that tips a
turn over rather than something the loop happened to survive.

Acceptance criteria

  • read_manifest caps its serialised result at a byte budget and marks a
    truncated result in a way the model can act on (same contract shape as
    read_manifest_rows: a flag plus the true total)
  • The truncation is described in the tool's own description, so the model
    knows to follow up with read_manifest_rows
  • The tool descriptions that currently say "ALWAYS call read_manifest first"
    point at read_manifest_meta instead, which is what they actually need
    (writeable rules and shape, not rows)
  • A card whose data fits under the budget returns byte-identically to today
  • Coverage: a card seeded past the budget returns a marked, bounded result;
    a small card is unchanged

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:chatChat widget + gateway integration + voiceenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions