Skip to content

Fix CreditResponse (0x8009) decode crashing the connection - #22

Open
mjquinlan2000 wants to merge 2 commits into
rabbitmq-community:mainfrom
gigsmart:fix/credit-response-decode
Open

Fix CreditResponse (0x8009) decode crashing the connection#22
mjquinlan2000 wants to merge 2 commits into
rabbitmq-community:mainfrom
gigsmart:fix/credit-response-decode

Conversation

@mjquinlan2000

Copy link
Copy Markdown

Problem

RabbitMQStream.Message.Decoder lists :credit among the correlated responses, so it decodes a CreditResponse by reading correlation_id::32, code::16 (6 bytes). But a CreditResponse has no correlation id — its body is response_code::16, subscription_id::8 (3 bytes). Decoding one therefore raises MatchError, and because Buffer.parse_frames/2 calls Decoder.decode/1 directly inside the connection GenServer's handle_info with no rescue, that single frame crashes the whole connection.

The broker sends a CreditResponse only on error — e.g. crediting a subscription it no longer knows about — which happens routinely during single-active-consumer rebalancing and after a reconnect. In production this crash-looped a consumer under load (each reconnect re-hit the frame).

Protocol references

Per the RabbitMQ streams protocol spec — deps/rabbitmq_stream/docs/PROTOCOL.adoc:

Credit is not a correlated command — the request carries a SubscriptionId, not a CorrelationId:

Credit => Key Version SubscriptionId Credit
  Key => uint16 // 0x0009
  Version => uint16
  SubscriptionId => uint8
  Credit => uint16 // the number of chunks that can be sent

And the response has no CorrelationId — only a response code and subscription id:

CreditResponse => Key Version ResponseCode SubscriptionId
  Key => uint16 // 0x8009
  Version => uint16
  ResponseCode => uint16
  SubscriptionId => uint8

The spec also notes the response is an error-only signal:

NB: the server sent a response only in case of problem, e.g. crediting an unknown subscription.

A real frame seen in production was <<0,0,0,7, 0x80,0x09, 0,1, 0x00,0x11, 0x07>> — length 7, key 0x8009, version 1, ResponseCode = 0x0011 (precondition_failed), SubscriptionId = 7. The correlated-response clause tried to read a 4-byte correlation id from the 3-byte body → MatchError.

Fix

  1. Correctness — a dedicated :credit decode/2 clause reads code::16, subscription_id::8 (no correlation id) and returns the existing %Types.CreditResponseData{}; :credit is removed from the correlated-response clause's command list.
  2. Resilience (defense in depth)Buffer.parse_frames/2 wraps the per-frame Decoder.decode/1 in try/rescue: an undecodable frame is logged (Logger.warning) and skipped instead of crashing the connection process. This keeps a single malformed/unexpected frame from taking down the whole connection.

Tests

Adds an offline test/message/decoder_test.exs (no broker required):

  1. The production CreditResponse bytes decode to %Response{command: :credit, code: :precondition_failed} with correlation_id: nil and data: %Types.CreditResponseData{}.
  2. parse_frames/2 drops an unknown/undecodable frame while a valid frame in the same batch still decodes.

The existing broker-tagged integration tests are unaffected.

The decoder listed :credit among correlated responses and tried to read a
4-byte correlation_id, but CreditResponse has none (code + subscription_id
only), raising MatchError and crashing the connection. Decode it directly.
Wrap per-frame Decoder.decode in try/rescue: log and drop a frame that
fails to decode instead of crashing the connection GenServer. Defense in
depth so no single unexpected frame can take down the stream pipeline.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants