What did you expect to see?
Stable height advancement and finalization on a small shard (3 validators + observer) over multi-day runs. Specifically, we expected:
- Height advancement to stay roughly constant, ideally around 3–8 seconds
- Finalization delay to stay roughly constant, ideally around 10–20 seconds
Multi-parent merge to operate normally as the main code path, since this is the core architectural advantage of f1r3node's Casper consensus.
What did you see instead?
Severe degradation of both height advancement and finalization over time, accompanied by extensive use of a single-parent fallback path that appears to disable multi-parent merge in practice.
Steps to reproduce the bug
- Start a shard of 3 validators + 1 observer node
- Enable heartbeat (check-interval = 3 seconds; max-lfb-age = 5 seconds; self-propose-cooldown = 6 seconds)
- Let the network run for 72+ hours.
- Observe the metrics over time: height advancement, finalization delay (block-processed → finalized), and the rate of compute_parents_post_state fallback ... reason=merge_scope_too_large warnings in the logs.
Degradation becomes visible after ~30 hours of continuous operation and continues to worsen until at least day 4 (longest run we have).
What version of F1R3FLY Node are you using?
rust v0.4.13 with changed hardcoded parameters: FRONTIER_CHASE_MAX_LAG=2 and STALE_RECOVERY_MIN_INTERVAL_MS=5_000
What is your environment?
- 3 validators + 1 observer running as Docker containers
- validator1 + validator2: co-located on the same VM
- validator3: separate VM
- observer: separate VM
What did you expect to see?
Stable height advancement and finalization on a small shard (3 validators + observer) over multi-day runs. Specifically, we expected:
Multi-parent merge to operate normally as the main code path, since this is the core architectural advantage of f1r3node's Casper consensus.
What did you see instead?
Severe degradation of both height advancement and finalization over time, accompanied by extensive use of a single-parent fallback path that appears to disable multi-parent merge in practice.
Steps to reproduce the bug
Degradation becomes visible after ~30 hours of continuous operation and continues to worsen until at least day 4 (longest run we have).
What version of F1R3FLY Node are you using?
rust v0.4.13 with changed hardcoded parameters: FRONTIER_CHASE_MAX_LAG=2 and STALE_RECOVERY_MIN_INTERVAL_MS=5_000
What is your environment?