Skip to content

Multi-parent merge degrades over time: merge_scope_too_large fallback dominates after 24h, slot time grows 8×, finalization 11× #500

Description

@Timekiller7

What did you expect to see?

Stable height advancement and finalization on a small shard (3 validators + observer) over multi-day runs. Specifically, we expected:

  • Height advancement to stay roughly constant, ideally around 3–8 seconds
  • Finalization delay to stay roughly constant, ideally around 10–20 seconds

Multi-parent merge to operate normally as the main code path, since this is the core architectural advantage of f1r3node's Casper consensus.

What did you see instead?

Severe degradation of both height advancement and finalization over time, accompanied by extensive use of a single-parent fallback path that appears to disable multi-parent merge in practice.

Steps to reproduce the bug

  1. Start a shard of 3 validators + 1 observer node
  2. Enable heartbeat (check-interval = 3 seconds; max-lfb-age = 5 seconds; self-propose-cooldown = 6 seconds)
  3. Let the network run for 72+ hours.
  4. Observe the metrics over time: height advancement, finalization delay (block-processed → finalized), and the rate of compute_parents_post_state fallback ... reason=merge_scope_too_large warnings in the logs.

Degradation becomes visible after ~30 hours of continuous operation and continues to worsen until at least day 4 (longest run we have).

What version of F1R3FLY Node are you using?

rust v0.4.13 with changed hardcoded parameters: FRONTIER_CHASE_MAX_LAG=2 and STALE_RECOVERY_MIN_INTERVAL_MS=5_000

What is your environment?

  • 3 validators + 1 observer running as Docker containers
  • validator1 + validator2: co-located on the same VM
  • validator3: separate VM
  • observer: separate VM

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions