CompactionEdit: server-side tool uses are cleared at stale message indices when keep_tool_inputs=False removes messages

[ebarkhordar]

Description

CompactionEdit collects client-side and server-side tool uses in a single enumeration of the message list (_compaction/edit.py, Phase 2), recording each one's positional message index. When keep_tool_inputs=False, the client-side pass removes tool messages entirely (del result[tool_idx]), which shifts every later message down.

The client-side pass protects its own indices by iterating in reverse. The server-side indices come from the same pre-deletion enumeration but are consumed afterwards, and nothing rebases them. So with keep_tool_inputs=False plus any server-side tool use (for example web_search) that sits after a cleared client-side tool call, the server-side clearing addresses the wrong message.

Two outcomes, depending on what shifted into the stale index:

1. If it lands on a message with string content, compaction raises IndexError: list assignment index out of range.
2. If it lands on an assistant message with list content, nothing raises. The tool use is written into that unrelated message, overwriting whatever content occupied that position, and the tool use that was meant to be cleared keeps its result.

The second case looks like the more serious one: the history sent to the model then contains a tool use the assistant never made, and compaction silently does not compact the entry it selected.

Steps to reproduce

Verified on 8117bf2 in a clean python:3.12-slim container, pip install -e ".[dev]".

import asyncio
from inspect_ai._util.content import ContentText, ContentToolUse
from inspect_ai.model import ChatMessageAssistant, ChatMessageTool, ChatMessageUser
from inspect_ai.model._compaction.edit import CompactionEdit
from inspect_ai.model._model import get_model
from inspect_ai.tool import ToolCall

ws = ContentToolUse(tool_type="web_search", id="ws1", name="web_search",
                    arguments='{"query": "q"}', result='[{"title": "Result"}]')

messages = [
    ChatMessageUser(content="Start"),
    ChatMessageAssistant(content="Using tool", tool_calls=[ToolCall(id="t1", function="bash", arguments={})]),
    ChatMessageTool(content="out", tool_call_id="t1", function="bash"),
    ChatMessageAssistant(content=[ContentText(text="Searching"), ws]),
    ChatMessageUser(content="Follow up"),
]

strategy = CompactionEdit(keep_thinking_turns="all", keep_tool_uses=0, keep_tool_inputs=False)
asyncio.run(strategy.compact(get_model("mockllm/model"), messages, []))

Expected behavior

The server-side tool use is cleared in the message that carried it.

Actual behavior

IndexError: list assignment index out of range

raised from content_list[content_idx] = cleared_tool_use in _apply_server_tool_clearing.

With keep_tool_inputs=True (the default) the same history compacts cleanly, since no message is removed and no index shifts.

For the silent variant, adding a second client-side tool call before the web_search message and a later assistant message with list content produces no exception: the cleared tool use is written into that later message instead, and the original keeps its result.

Environment

• inspect_ai at commit 8117bf2 (also present since #3120, which added server-side tool compaction)
• Python 3.12, Linux

I have a fix and tests ready and will open a PR against this issue shortly. Happy to take a different approach if you would rather handle it another way.

Filed with AI assistance (the repro above was executed, not inferred).