Context
PIP-468 allows consumer membership changes to recompute assignments without changing the segment layout. Different ScalableConsumerAssignment messages can therefore carry the same layout_epoch.
The current protocol carries the layout epoch and segment list, but no assignment generation, controller term, or session incarnation. The Java V5 client rejects only epoch < currentEpoch, so changed equal-epoch assignments are deliberately accepted.
Within one live ClientCnx, wire order gives clients an ordering. Across controller reconnect or leader failover, I could not find a protocol-level fencing rule.
There is another ordering detail in the current broker path: registerConsumer can push an assignment update before ServerCnx writes the subscribe response, so the response does not appear to be an obvious synchronization barrier.
Questions
- Within one controller connection, should clients apply changed equal-epoch assignments strictly in receive order?
- Is
CommandScalableTopicSubscribeResponse a baseline that precedes all pushed updates, or may a push received before the response represent newer state?
- After reconnect or controller leader failover, can a callback or frame from the previous connection race the new connection's initial assignment?
- Should clients locally fence all callbacks by connection incarnation?
- Is an assignment generation or controller/session term planned for the wire protocol?
Concrete example: assignments A and B both have layout_epoch = 7, but differ because a consumer joined. If A comes from the old connection and B from the new connection, is there any protocol-supported way to determine which is authoritative?
References: PIP-468, SubscriptionCoordinator.computeAssignment, ScalableConsumerClient.onAssignmentUpdate, and ServerCnx.handleCommandScalableTopicSubscribe on v5.0.0-M1 and current master.
Context
PIP-468 allows consumer membership changes to recompute assignments without changing the segment layout. Different
ScalableConsumerAssignmentmessages can therefore carry the samelayout_epoch.The current protocol carries the layout epoch and segment list, but no assignment generation, controller term, or session incarnation. The Java V5 client rejects only
epoch < currentEpoch, so changed equal-epoch assignments are deliberately accepted.Within one live
ClientCnx, wire order gives clients an ordering. Across controller reconnect or leader failover, I could not find a protocol-level fencing rule.There is another ordering detail in the current broker path:
registerConsumercan push an assignment update beforeServerCnxwrites the subscribe response, so the response does not appear to be an obvious synchronization barrier.Questions
CommandScalableTopicSubscribeResponsea baseline that precedes all pushed updates, or may a push received before the response represent newer state?Concrete example: assignments A and B both have
layout_epoch = 7, but differ because a consumer joined. If A comes from the old connection and B from the new connection, is there any protocol-supported way to determine which is authoritative?References: PIP-468,
SubscriptionCoordinator.computeAssignment,ScalableConsumerClient.onAssignmentUpdate, andServerCnx.handleCommandScalableTopicSubscribeonv5.0.0-M1and currentmaster.