If blocks are struggling to be closed or otherwise have items added to them in a timely manner, we should add some sort of metrics or at least logging to indicate which block(s) are impacted and for how long.
Threshold: If no item has been added to the block for 1.5x the block duration OR the block is not closed after 2x block duration since it was opened
Action: log a warning message when the issue is detected once at WARN, and then if the block recovers, log a message at INFO indicating recovery and the time it took. We may want to also add a metric that gets incremented when we encounter these slow operations.
If blocks are struggling to be closed or otherwise have items added to them in a timely manner, we should add some sort of metrics or at least logging to indicate which block(s) are impacted and for how long.
Threshold: If no item has been added to the block for 1.5x the block duration OR the block is not closed after 2x block duration since it was opened
Action: log a warning message when the issue is detected once at WARN, and then if the block recovers, log a message at INFO indicating recovery and the time it took. We may want to also add a metric that gets incremented when we encounter these slow operations.