We learnt a lot about what a SinkCluster should look like while writing CassandraSinkCluster, as a result there is a lot of confusing invariants to uphold and it is very difficult to understand.
In comparison, KafkaSinkCluster, which was written later, is a lot nicer.
Here are some things we can take from KafkaSinkCluster to improve the design of CassandraSinkCluster.
- Keep the control connection as purely a control connection, do not direct its responses to the client by default
- Combine rewrite_requests with route_requests
- We need to be handling the effects of requests in the order they were received in, but with these two stages, sometimes the effect is handled in rewrite_requests and sometimes in route_requests, this causes particular issues with use statements.
- Maybe split up sending requests into route_requests -> send_requests stages, need to check if this actually brings value for CassandraSinkCluster.
We learnt a lot about what a SinkCluster should look like while writing CassandraSinkCluster, as a result there is a lot of confusing invariants to uphold and it is very difficult to understand.
In comparison, KafkaSinkCluster, which was written later, is a lot nicer.
Here are some things we can take from KafkaSinkCluster to improve the design of CassandraSinkCluster.