Skip to content

Scope Bedrock Responses API continuation state to a session #300

Description

@foundev

Summary

BedrockClient is shared across sessions and currently owns one content-keyed Responses API continuation cache. The key is based on the input message prefix and does not include a conversation/session identity or verify the echoed assistant response. Two sessions with identical initial contexts can therefore contend for the same cache entry and potentially continue from the wrong server-side response ID.

This was identified by inspection of master at afeec142a24dac622a7ff6f54d0f5021f319fb17.

Relevant code

  • src/main.rs:718-746 shares the backend across sessions.
  • src/bedrock_client.rs:370-394 stores the process-wide responses_chain cache.
  • src/bedrock_client.rs:495-568 hashes and looks up message prefixes.
  • src/bedrock_client.rs:1149-1154 stores the response ID by content hash.

Expected behavior

A continuation must use only a response ID produced for the same logical ACP conversation and compatible model/request configuration.

Suggested approach

Prefer making response-chain state session/conversation-owned and explicitly passing the prior response ID. If the cache remains in BedrockClient, include a stable conversation identity and request-shaping inputs and validate the assistant turn associated with the cached response.

Acceptance criteria

  • Concurrent sessions with identical first-turn input but different model responses continue from their own response IDs.
  • Changing the model or tool schema cannot reuse an incompatible continuation entry.
  • Tests cover concurrent divergent responses and separate second turns.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions