sendMessage treats every rd_kafka_produce failure the same (src/kafka/producer.zig:264-268). RD_KAFKA_RESP_ERR__QUEUE_FULL is a transient condition — the local queue is simply ahead of the broker — but it propagates as an error and the process exits.
No data is lost (the LSN is not confirmed, the batch is re-sent after restart) and each cycle makes progress, so under the current fail-fast policy this is acceptable. But under sustained bursts it means repeated restarts where a short blocking retry would do, and it will cap throughput against the Debezium-parity goal.
Ask: on __QUEUE_FULL, call rd_kafka_poll to drain delivery reports and retry the produce with a bounded backoff before giving up.
sendMessagetreats everyrd_kafka_producefailure the same (src/kafka/producer.zig:264-268).RD_KAFKA_RESP_ERR__QUEUE_FULLis a transient condition — the local queue is simply ahead of the broker — but it propagates as an error and the process exits.No data is lost (the LSN is not confirmed, the batch is re-sent after restart) and each cycle makes progress, so under the current fail-fast policy this is acceptable. But under sustained bursts it means repeated restarts where a short blocking retry would do, and it will cap throughput against the Debezium-parity goal.
Ask: on
__QUEUE_FULL, callrd_kafka_pollto drain delivery reports and retry the produce with a bounded backoff before giving up.