Vision embeddings dropped when an image is prefilled after a RAM session-cache restore (thinking enabled)

[GlossityLabs]

mtplx-vision-cache-issue.md

[pyMoLt]

Summary

With the x-mtplx-client request identified as OpenCode (either via the explicit header or via User-Agent sniffing in _request_client_hint_from_request), any request whose message history contains a prior tool result (tool_result_history_present=True) is routed through _should_force_clone_session_cache_for_opencode_tool_history(), which forces storage_restore_mode: "clone" by default (MTPLX_OPENCODE_TOOL_HISTORY_LIVE_FRONTIER and MTPLX_OPENCODE_TOOL_HISTORY_SESSIONBANK_BYPASS are both off by default). This clone-restore is itself a cache restore operation, which is exactly the trigger condition described in #296 ("Vision embeddings dropped when an image is prefilled after a RAM session-cache restore, thinking enabled").

In practice: once a tool that returns an image (e.g. a browser-screenshot tool, or any read of a local image file) has been called once in an OpenCode session, every subsequent image in that session is at risk of landing inside a force-clone-restored span and getting silently dropped from the KV cache — the model then hallucinates a plausible-but-wrong image description, exactly matching the symptom in #296 ("a fluffy white cat" etc. — in our case: describing an entirely unrelated stock photo, or in one case ignoring both the image and the actual question and answering a generic greeting instead).

Why this looks OpenCode-specific from the outside

_should_force_clone_session_cache_for_opencode_tool_history() calls _is_opencode_client() directly — it is not gated behind the broader _TOOL_CONTRACT_AGENT_CLIENT_HINTS = ("opencode", "pi", "hermes") tuple that other agent-contract behavior uses. Pi and Hermes clients never hit this specific forced-clone-restore path for tool-history requests, so the same underlying #296 restore-vision-drop bug is far less likely to surface for them even though the root cause (in the vision/restore path) presumably isn't OpenCode-specific at all. This is why "it works fine in Pi against the same server" is a real, valid observation and not evidence against #296 — it's evidence that OpenCode's traffic is far more likely to hit the #296 trigger condition than Pi's is, because of this OpenCode-only forced-clone-restore behavior.

What we ruled out first
Not a header/session-ID issue: identical non-streaming and streaming curl requests against /v1/chat/completions with the same x-mtplx-session-id, with and without x-mtplx-client: opencode, all correctly described a test image (single-turn and simple 3-message multi-turn cases).
Not fixable by omitting x-mtplx-client: _request_client_hint_from_request() falls back to sniffing "opencode" out of the literal HTTP User-Agent header when the explicit header is absent, so client detection (and therefore the forced-clone-restore behavior) doesn't actually change either way for OpenCode's own HTTP client.
Not the OpenCode CLI/Desktop version: reproduced on both an outdated (1.3.15) and current (1.18.21) OpenCode CLI build, and independently in OpenCode Desktop (which runs its own bundled server, not the CLI binary at all).
Not a broken screenshot capture: verified via file size and separately via direct read of the saved PNG that the image itself is intact on disk when the bug occurs.
What we haven't fully confirmed yet
Whether MTPLX_OPENCODE_TOOL_HISTORY_LIVE_FRONTIER=1 avoids the vision drop while keeping cache reuse (testing now).
Exact reproduction steps with request-level telemetry (storage_restore_mode, boundary_restore, first_image_pad_position etc. from request-log-8000.jsonl, per the format used in #296) for an OpenCode-originated request specifically — happy to capture and attach this once we can reproduce reliably on demand rather than intermittently in normal usage.
Environment
MTPLX: 2.9.1
Model: Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed-FP16 (and -Quality-FP16)
Client: OpenCode Desktop 1.18, provider config @ai-sdk/openai-compatible, reasoning: true, reasoningEffort: xhigh
Hardware/OS: Macbook Pro M3 Max

Suggested fix direction
Given _opencode_prompt_contract_system_prompt() already deliberately avoids touching the system prompt ("OpenCode owns its agent/system instructions"), it seems consistent with that philosophy for the forced-clone-restore behavior to either (a) not apply when the restored span contains an image, falling back to a fresh prefill for just that turn, or (b) default MTPLX_OPENCODE_TOOL_HISTORY_LIVE_FRONTIER on for image-bearing requests specifically, rather than requiring a global env var most users won't know exists.

[youssofal]

Fixed in the next release. Your telemetry split was the tell: every failing row was the near restore lane, and that lane had no vision splice plumbing at all, so after a warm restore the suffix containing the image pads went in as plain ids. The splice is now threaded end to end on that path with a guard that fails loudly instead of silently dropping the image.

pyMoLt, the clone mode routing you describe is a separate mechanism from this bug. If you are seeing bad behavior from it, open a dedicated issue and I will look at it on its own.

[pyMoLt]

Update: reproduces on Pi client too — not OpenCode-specific, and RAM-cache pruning doesn't fix it

Following up on the forced-clone-restore analysis above: the same vision-drop symptom now reproduces on the Pi coding agent client against the same MTPLX server, with x-mtplx-client/request_client_hint resolving to "pi" and opencode_tool_history_force_clone_restore: false throughout — i.e. entirely outside the OpenCode-exclusive forced-clone-restore path discussed above. This confirms the underlying #296 mechanism (reasoning-on + cache restore → vision embeddings dropped) is genuinely client-agnostic; OpenCode's forced-clone behavior just makes it more likely to trigger, it isn't the root cause.

Additional context: the MTPLX session bank was fully cleared before this test run. A sequence of several unrelated tasks was then run in the same session, with the image-download-and-describe task deliberately placed last. The failure happened on that final task — i.e. not because of stale cross-session cache, but because enough restore activity had accumulated within the single ongoing session by the time the image turn came up (consistent with the ssd_clone restore observed in the log entry below, since the RAM tier had also been pruned beforehand).

New finding: pruning the RAM session cache does not fix it — it just moves the trigger to the SSD tier.

After fully pruning the RAM session cache (hoping to avoid any restore), the very next image-bearing turn in the same session still failed — but this time via an SSD-tier restore instead of a RAM one. Request-log entry for the failing request:

• request_id: chatcmpl-a1f7f54dbb2e4b8695fc662bec705633
• session_id: anon-790c2800160aad58
• request_client_hint: "pi"
• session_cache_hit: true
• session_restore_mode: "ssd_clone"
• cache_source: "ssd"
• ssd_cache_hit: true, ssd_restore_s: ~4.85
• cached_tokens: 7619, new_prefill_tokens: 7769, prompt_tokens: 15388
• request_vision_images: 1, first_image_pad_position: 13980
• request_reasoning_effort: "xhigh", request_enable_thinking: true, preserve_thinking_effective: true
• opencode_tool_history_force_clone_restore: false, opencode_tool_history_live_frontier_restore: false

The model's response described a generic, plausible-sounding "official presidential portrait" that did not match the actual attached image — the classic #296 hallucination pattern, just now confirmed happening via the cold/SSD storage tier (storage_restore_mode: "clone", lazy_kv: false) rather than the RAM tier.

Implication: the fix for #296 likely needs to cover the SSD/cold-tier restore path (session_restore_mode: "ssd_clone") in addition to the RAM-tier clone / block_prefix_clone variants already discussed, since evicting the RAM cache simply pushes affected sessions down to the next restore tier rather than avoiding a restore altogether. A full RAM+SSD cache wipe is being tested next to see whether the bug still reproduces when both tiers start genuinely cold — will report back with the result.

[pyMoLt]

pyMoLt, the clone mode routing you describe is a separate mechanism from this bug. If you are seeing bad behavior from it, open a dedicated issue and I will look at it on its own.

Thanks, it seems to be the same bug from my understanding; so no different issue. Thanks for the great work on the project!!