I have a specific use case where I have an append-only distributed set that is constantly being added to.
A gossip layer will keep all servers in sync. Items could still be lost due to reboots, network partitions etc.
A new async-friendly version of the RIBLT structure will be created.
It does not make sense to build the RIBLT of the entire set, as it will be changing too quickly. Items that look like they have been missed might still be in transit on the gossip layer.
Internally, this will be implemented as two streams of CodedSymbols. One for 'old' items and one for 'new' items.
A threshold time decides if the items go to the old or new stream.
The 'old' should be stable and useful for exchange between servers to detect items that were missed.
Every minute or so, the 'new' will be joined to the 'old' and an empty 'new' created. The threshold time will be updated to a new time.
I have a specific use case where I have an append-only distributed set that is constantly being added to.
A gossip layer will keep all servers in sync. Items could still be lost due to reboots, network partitions etc.
A new async-friendly version of the RIBLT structure will be created.
It does not make sense to build the RIBLT of the entire set, as it will be changing too quickly. Items that look like they have been missed might still be in transit on the gossip layer.
Internally, this will be implemented as two streams of CodedSymbols. One for 'old' items and one for 'new' items.
A threshold time decides if the items go to the old or new stream.
The 'old' should be stable and useful for exchange between servers to detect items that were missed.
Every minute or so, the 'new' will be joined to the 'old' and an empty 'new' created. The threshold time will be updated to a new time.