Skip to content

IBD: count only requested blocks in the download window - #953

Merged
bkeroack merged 21 commits into
release/0.6from
fix/ibd-window-accounting
Oct 14, 2026
Merged

bkeroack merged 21 commits into
release/0.6from
fix/ibd-window-accounting

Conversation

@bkeroack

@bkeroack bkeroack commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

IBD download window. IbdScheduler::block_received (node/src/net/ibd.rs) recorded any delivered block whose hash matched a height as downloaded, including heights the connector had already passed. The Duplicate arm of handle_block_ibd calls it for every re-sent block. downloaded drains only as the cursor advances (connect_cursor_advanced), so those entries stayed in the -maxahead window, and the near-cursor priority zone in assign_blocks_up_to was gated by the same window. With the window full and the next height not in flight (released from a peer that did not deliver it, or after a restart with a smaller -maxahead), no peer was ever given that height and the download stopped. Core measures its window from the last common block, so the blocks just above the tip are always inside it (net_processing.cpp:1436-1441).

Any delivered block also reset the sender's stall clock and failure count, so a peer still owing heights looked active. Core clears m_stalling_since only when a requested block arrives (RemoveBlockRequest, net_processing.cpp:1199-1230).

A released height that waited in the pending pool while its block arrived late and was connected from disk could also be handed out again after the cursor passed it.

Local store errors charged to the sender. handle_block_ibd added 10 to the sender's ban score for every store_block error except an unknown parent, and store_bg_block added 5. A block-file write or fsync failure (ChainError::FlatFile) or a database write failure (ChainError::Storage) is this node's problem, so after the disk fills each peer reached the ban threshold after ten blocks. Core treats a failed write as fatal (FatalError in AcceptBlock) and does not call Misbehaving.

Tip-following sweep. With no scheduler (the gap to the best header is 24 blocks or less), the run loop called request_missing_blocks for every serving peer on each tip advance and every 5 s. Each peer got the same list of up to 128 missing blocks, so each block was downloaded once per peer. Core keeps one request per block (mapBlocksInFlight).

Which peers are asked. Forward IBD (assign_peer_work) and the AssumeUTXO background download assigned blocks to any connected peer, including ones that advertise neither NODE_NETWORK nor NODE_NETWORK_LIMITED. Core's CanServeBlocks (net_processing.cpp:1152-1155) excludes them. Core also asks a limited peer only for blocks within 286 of its tip (:1533), never during IBD (:6164), and never for historical blocks (:6176). The background download had no cap on what a peer had outstanding (16 new heights per peer every 2 s pass), restarted its round-robin at the same peer each pass, could give a timed-out height straight back to the same peer, and ignored notfound.

Fix

  • block_received counts only a height that is in flight, and records it as downloaded only when it is above the connect cursor. The peer's activity clock, failure count and blocks_received move only when the block was in flight to that peer. Delivery by a different peer still frees the holder's slot, as before.
  • The priority zone (cursor+1..cursor+256) is exempt from max_ahead, so the window can be exceeded by at most 256 blocks. The pending pool drops heights at or below the cursor.
  • store_error_is_the_blocks: only Validation and CheckpointMismatch are scored, on both paths. Other errors are logged at warn and the sender is not charged.
  • sweep_missing_blocks replaces the per-peer loop in the run-loop sweep. Each missing block goes to one peer, at most 16 per peer per pass. A block asked of anyone (as a full block) less than 30 s ago is skipped, and an older one goes to a peer not asked before when there is one. request_missing_blocks and its other callers are unchanged, as are compact-block requests.
  • PeerInfo::advertises_blocks (Core's CanServeBlocks) and PeerInfo::is_limited (Core's IsLimitedPeer). Forward IBD and the sweep require advertises_blocks. A limited peer gets IBD work only when every height it could be given is within 286 of its tip. The background download uses NODE_NETWORK peers only.
  • BgDownloader::assign caps outstanding heights at 16 per peer (Core's MAX_BLOCKS_IN_TRANSIT_PER_PEER), moves the round-robin start on each pass, and gives a released height (timed out, or answered with notfound) to a different peer unless that peer is the only one. The notfound handler releases background heights asked of that peer.
  • Operator Manual (ibd.md): the window always fetches the 256 blocks above the tip, and which peers are asked for blocks.

Tests

New files: node/src/net/ibd_ibdwindow_tests.rs, bg_catchup_ibdwindow_tests.rs, manager_ibdwindow_tests.rs.

To check each test, I reverted ibd.rs to the parent and disabled each guard in bg_catchup.rs and manager.rs. All 14 tests in the first commit then failed on their own assertions:

  • a_re_sent_connected_block_does_not_count_as_downloaded: fails without the fix: yes ("connected heights entered the window", left 3). It also checks that a requested block still counts.
  • an_unrequested_block_above_the_cursor_does_not_count: yes.
  • a_full_window_still_hands_out_the_next_height: yes ("height 1 not handed out with the window full: []").
  • a_released_next_height_goes_to_another_peer_with_the_window_full: yes ("height 1 not given to peer 2: []").
  • a_pooled_height_below_the_cursor_is_not_handed_out: yes ("connected heights handed out: [2, 3, 4, …]").
  • only_a_requested_block_resets_the_stall_clock: yes ("an unrequested block reset the failure count"). It also checks that the peer's own block does reset it.
  • a_local_write_failure_does_not_charge_the_sender: yes ("forward IBD: the sender was charged for our disk", left 10). It makes the block files read-only, and checks that a block with a bad merkle root is still charged 10 (forward) or 5 (background).
  • ibd_work_goes_only_to_peers_that_can_serve_it: yes ("no block service: asked for […]"). It also checks that NODE_NETWORK and a recent limited peer get work. The 286-block limited case was not perturbed on its own.
  • the_sweep_asks_one_peer_per_block: yes ("asked for 2 times"). It also checks that an aged request goes to a different peer.
  • the_sweep_asks_a_peer_that_has_the_block: yes, with the height filter removed ("a peer without the blocks was asked"). It also checks that a limited peer is asked only for blocks it keeps, and that a notfound frees the block for the next pass.
  • notfound_releases_a_background_height: yes.
  • bg_catchup: a_peer_at_the_cap_gets_nothing_more_on_the_next_pass, a_released_height_goes_to_a_different_peer, the_round_robin_start_moves_each_pass, notfound_releases_only_the_askers_height: yes, each.

Local gate: clippy (all features, and node with no default features), cargo test -p node --lib, and cargo test -p satd --test regtest -- ibd sync.

Notes

  • Stacked on p2p: bound learned addresses and outbound dials; manual peers outside the automatic cap #952 (merge that first).
  • The sweep's 30 s re-request delay is shorter than Core's block download timeout. The sweep used to re-send every 5 s, and a request that went unanswered for much longer would leave a block near the tip waiting.
  • Not changed: the single-peer cooldown bypass in IbdScheduler (a lone slow peer is re-asked for the heights it still owes every 15 s), and the re-created scheduler not reusing the startup scan of blocks already on disk.

Release notes

IBD no longer stalls when the -maxahead window is full while the next block is not being downloaded. Blocks re-sent for heights that are already connected no longer count against the window. A peer that is still owed blocks is only treated as active when it delivers blocks it was asked for. A local disk or database write failure while storing a downloaded block no longer raises the sending peer's ban score. When following the tip, satd asks one peer for each missing block instead of all of them. Block downloads, including the AssumeUTXO background download, now go only to peers that advertise block service, as in Bitcoin Core.

🤖 Generated with Claude Code

bkeroack and others added 20 commits October 7, 2026 16:18
…ds per peer

- v1 frames are capped at Core's MAX_PROTOCOL_MESSAGE_LENGTH (4,000,000
  bytes) instead of 32 MiB. The v1 and v2 receive buffers grow as payload
  bytes arrive, at most 256 KiB ahead of them (V1Transport::readData,
  V2Transport MAX_RESERVE_AHEAD), and a large v1 buffer is released after
  its message.
- inv/getdata over MAX_INV_SZ (50,000), headers over MAX_HEADERS_RESULTS
  (2000) and addr/addrv2 over MAX_ADDR_TO_SEND (1000) are misbehaviour, as
  in Core; a notfound over 5016 entries is ignored.
- A peer's queue counts its bytes. getdata is served as Core's
  ProcessGetData does: entries are kept in a per-peer backlog and served
  until the queue passes 1,000,000 bytes (-maxsendbuffer's default), and
  the rest as it drains. The peer's next message is not read while a
  getdata of its is being served or its queue is over the limit
  (ProcessMessages). -maxuploadtarget is charged only for blocks queued.
- The write loop watches the disconnect signal during a write, and drops
  a peer whose socket takes no bytes for 20 minutes (TIMEOUT_INTERVAL,
  "socket sending timeout"). Sent bytes are counted as the socket takes
  them.
- A version whose user agent is over MAX_SUBVERSION_LENGTH is dropped, as
  Core's LIMITED_STRING read does; the user agent and unknown message
  types are sanitized with Core's SanitizeString before they are stored
  or logged.
- getcfilters/getcfheaders/getcfcheckpt requests that Core's
  PrepareBlockFilterRequest refuses now disconnect instead of being
  ignored.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The write loop reads nothing more from a peer until its getdata has been
served, so the manager's 500 ms drain tick bounded a peer fetching blocks
to about one getdata, or one send buffer's worth of a large one, per tick.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Deal with a peer whose misbehaviour reaches the threshold as Core's
MaybeDiscourageAndDisconnect does (net_processing.cpp), in its order:
a noban peer and a manual connection are not punished; a peer on a
local address (CNetAddr::IsLocal: 127.0.0.0/8, 0.0.0.0/8, ::1) is
disconnected without punishing the address; anyone else is
disconnected and its address banned for -bantime, as before (Core
discourages it).

satd banned every peer but noban ones. Every inbound onion peer arrives
from the address Tor dialled from, usually 127.0.0.1, as does every
unwhitelisted local client, so one such peer's misbehaviour refused all
of them until the ban expired, across restarts. A manual peer was
banned and then skipped by the redial loop.

Peers accepted on the onion listener are now marked (Core's
m_inbound_onion), and getpeerinfo reports them on the onion network, as
Core's ConnectedThroughNetwork does. An outbound onion peer is banned by
its onion host, the address Core punishes, rather than by the 0.0.0.0
placeholder socket every onion peer shares.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
serve_compact_block reported a MSG_CMPCT_BLOCK entry as answered even
when its cmpctblock could not be queued, so the entry left the getdata
backlog with nothing sent. It now says whether the message was queued,
and an entry that was not stays first, as every other entry does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s from

Core decides the disconnect-only rung on the address alone
(pnode.addr.IsLocal(), net_processing.cpp), and Tor on the same host
dials from loopback. With Tor on another host (-bind=<lan address>=onion)
every onion peer arrives from that host's address. Core discourages it,
and still admits it while inbound slots are free (net.cpp); satd's ban
would refuse every onion peer for -bantime. An inbound onion peer is
now disconnected without a ban whatever address it arrives from.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the automatic cap

- Core's address rate limit (ProcessAddrs): a token bucket per peer, one
  token to start, 0.1 a second up to 1000, plus 1000 when the node sends
  getaddr; entries are shuffled first and the addr permission is exempt.
  getpeerinfo reports addr_processed and addr_rate_limited.
- A gossiped address joins the dial list only if the address book holds
  it. The list is a set in arrival order, capped at the book's size.
- reserve_dial checks a dial's limit and takes its place under one lock,
  counting dials in flight: automatic dials against the automatic target,
  added nodes at most 8 at a time (Core's semAddnode), -connect,
  -seednode and onetry without a limit.
- The reconnect tick redials manual peers whatever the automatic count,
  then fills the free automatic slots. Automatic dials skip banned
  addresses, onion hosts included; manual dials do not (Core's
  OpenNetworkConnection). Gossip naming a banned onion host is not kept.
- Manual backoff stops at 60 s; backoff is cleared only by a dial that
  connects, not by a new tip.
- addnode onetry is outside every limit and succeeds for a peer already
  connected or being dialled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on-manual' into fix/addr-list-and-dial-limits
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged with B2's own tip (7c8b6ed) as the base, since this branch already
carried it and release/0.6 now holds its squashed equivalent. No conflicts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ss's backoff goes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the backoff while checking membership

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… peer for blocks it can serve

The IBD scheduler recorded any re-sent block whose hash matched a height as
downloaded, including heights the connector had already passed. Those
entries drain only as the cursor advances, so they stayed in the
`-maxahead` window, and the near-cursor priority zone was gated by that
window: once it was full while the next height was not in flight, nothing
handed that height out again. Any delivered block also reset the sender's
stall clock. Now only an in-flight height above the cursor is recorded,
only a block asked of the peer resets its clock (Core's
RemoveBlockRequest), the priority zone is exempt from the window (Core's
window starts at the last common block), and a pooled height the cursor
has passed is dropped.

A failed block-file or database write while storing an IBD or background
block was charged to the sender (+10 / +5). Only a Validation or
CheckpointMismatch error is the block's fault now.

The tip-following sweep sent every missing block to every peer on each
pass. It now asks one peer per block, at most 16 per peer, and asks again
after 30 s, of a different peer when there is one.

Block download now requires NODE_NETWORK or NODE_NETWORK_LIMITED (Core's
CanServeBlocks); a limited peer gets IBD work only when every height it
could be given is within 286 of its tip, and the AssumeUTXO background
download uses NODE_NETWORK peers only. The background download caps what
is outstanding per peer at 16, rotates its starting peer, gives a released
height to a different peer, and releases a height on notfound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…eers are asked for blocks

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d frees a sweep request

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the sweep

The tip-following sweep now asks one peer per block, so it no longer asks
every peer for the same block.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bkeroack
bkeroack changed the base branch from fix/addr-list-and-dial-limits to release/0.6 October 14, 2026 02:12
Merged with B3's final head (c3b12e7) as the base, since this branch
already carried it and release/0.6 now holds its squashed equivalent.
No conflicts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bkeroack
bkeroack merged commit 6a18e0f into release/0.6 Oct 14, 2026
13 checks passed
@bkeroack
bkeroack deleted the fix/ibd-window-accounting branch October 14, 2026 02:13
bkeroack added a commit that referenced this pull request Oct 15, 2026
* p2p: apply Core's message size and count limits, and bound queued sends per peer

- v1 frames are capped at Core's MAX_PROTOCOL_MESSAGE_LENGTH (4,000,000
  bytes) instead of 32 MiB. The v1 and v2 receive buffers grow as payload
  bytes arrive, at most 256 KiB ahead of them (V1Transport::readData,
  V2Transport MAX_RESERVE_AHEAD), and a large v1 buffer is released after
  its message.
- inv/getdata over MAX_INV_SZ (50,000), headers over MAX_HEADERS_RESULTS
  (2000) and addr/addrv2 over MAX_ADDR_TO_SEND (1000) are misbehaviour, as
  in Core; a notfound over 5016 entries is ignored.
- A peer's queue counts its bytes. getdata is served as Core's
  ProcessGetData does: entries are kept in a per-peer backlog and served
  until the queue passes 1,000,000 bytes (-maxsendbuffer's default), and
  the rest as it drains. The peer's next message is not read while a
  getdata of its is being served or its queue is over the limit
  (ProcessMessages). -maxuploadtarget is charged only for blocks queued.
- The write loop watches the disconnect signal during a write, and drops
  a peer whose socket takes no bytes for 20 minutes (TIMEOUT_INTERVAL,
  "socket sending timeout"). Sent bytes are counted as the socket takes
  them.
- A version whose user agent is over MAX_SUBVERSION_LENGTH is dropped, as
  Core's LIMITED_STRING read does; the user agent and unknown message
  types are sanitized with Core's SanitizeString before they are stored
  or logged.
- getcfilters/getcfheaders/getcfcheckpt requests that Core's
  PrepareBlockFilterRequest refuses now disconnect instead of being
  ignored.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: wake the manager when a getdata or a getdata resume is handed over

The write loop reads nothing more from a peer until its getdata has been
served, so the manager's 500 ms drain tick bounded a peer fetching blocks
to about one getdata, or one send buffer's worth of a large one, per tick.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: don't ban local, inbound-onion or manual peers for misbehaviour

Deal with a peer whose misbehaviour reaches the threshold as Core's
MaybeDiscourageAndDisconnect does (net_processing.cpp), in its order:
a noban peer and a manual connection are not punished; a peer on a
local address (CNetAddr::IsLocal: 127.0.0.0/8, 0.0.0.0/8, ::1) is
disconnected without punishing the address; anyone else is
disconnected and its address banned for -bantime, as before (Core
discourages it).

satd banned every peer but noban ones. Every inbound onion peer arrives
from the address Tor dialled from, usually 127.0.0.1, as does every
unwhitelisted local client, so one such peer's misbehaviour refused all
of them until the ban expired, across restarts. A manual peer was
banned and then skipped by the redial loop.

Peers accepted on the onion listener are now marked (Core's
m_inbound_onion), and getpeerinfo reports them on the onion network, as
Core's ConnectedThroughNetwork does. An outbound onion peer is banned by
its onion host, the address Core punishes, rather than by the 0.0.0.0
placeholder socket every onion peer shares.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: keep a getdata cmpctblock entry that could not be queued

serve_compact_block reported a MSG_CMPCT_BLOCK entry as answered even
when its cmpctblock could not be queued, so the entry left the getdata
backlog with nothing sent. It now says whether the message was queued,
and an entry that was not stays first, as every other entry does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: tidy the MisbehaviourAction docs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: disconnect an inbound onion peer without a ban wherever Tor dials from

Core decides the disconnect-only rung on the address alone
(pnode.addr.IsLocal(), net_processing.cpp), and Tor on the same host
dials from loopback. With Tor on another host (-bind=<lan address>=onion)
every onion peer arrives from that host's address. Core discourages it,
and still admits it while inbound slots are free (net.cpp); satd's ban
would refuse every onion peer for -bantime. An inbound onion peer is
now disconnected without a ban whatever address it arrives from.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: bound learned addresses and outbound dials; manual peers outside the automatic cap

- Core's address rate limit (ProcessAddrs): a token bucket per peer, one
  token to start, 0.1 a second up to 1000, plus 1000 when the node sends
  getaddr; entries are shuffled first and the addr permission is exempt.
  getpeerinfo reports addr_processed and addr_rate_limited.
- A gossiped address joins the dial list only if the address book holds
  it. The list is a set in arrival order, capped at the book's size.
- reserve_dial checks a dial's limit and takes its place under one lock,
  counting dials in flight: automatic dials against the automatic target,
  added nodes at most 8 at a time (Core's semAddnode), -connect,
  -seednode and onetry without a limit.
- The reconnect tick redials manual peers whatever the automatic count,
  then fills the free automatic slots. Automatic dials skip banned
  addresses, onion hosts included; manual dials do not (Core's
  OpenNetworkConnection). Gossip naming a banned onion host is not kept.
- Manual backoff stops at 60 s; backoff is cleared only by a dial that
  connects, not by a new tip.
- addnode onetry is outside every limit and succeeds for a peer already
  connected or being dialled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: exempt a two-address test peer from the address rate limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: count only the announced addresses in the addr-fetch test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: setban of an onion host disconnects that onion peer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: a full address book replaces a terrible entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: a full network group replaces a terrible entry; a replaced address's backoff goes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: a full book of terrible tried entries takes new addresses; lock the backoff while checking membership

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* p2p: count only requested blocks in the IBD download window; ask each peer for blocks it can serve

The IBD scheduler recorded any re-sent block whose hash matched a height as
downloaded, including heights the connector had already passed. Those
entries drain only as the cursor advances, so they stayed in the
`-maxahead` window, and the near-cursor priority zone was gated by that
window: once it was full while the next height was not in flight, nothing
handed that height out again. Any delivered block also reset the sender's
stall clock. Now only an in-flight height above the cursor is recorded,
only a block asked of the peer resets its clock (Core's
RemoveBlockRequest), the priority zone is exempt from the window (Core's
window starts at the last common block), and a pooled height the cursor
has passed is dropped.

A failed block-file or database write while storing an IBD or background
block was charged to the sender (+10 / +5). Only a Validation or
CheckpointMismatch error is the block's fault now.

The tip-following sweep sent every missing block to every peer on each
pass. It now asks one peer per block, at most 16 per peer, and asks again
after 30 s, of a different peer when there is one.

Block download now requires NODE_NETWORK or NODE_NETWORK_LIMITED (Core's
CanServeBlocks); a limited peer gets IBD work only when every height it
could be given is within 286 of its tip, and the AssumeUTXO background
download uses NODE_NETWORK peers only. The background download caps what
is outstanding per peer at 16, rotates its starting peer, gives a released
height to a different peer, and releases a height on notfound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: the IBD window always fetches the blocks above the tip; which peers are asked for blocks

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* p2p: the sweep asks for a block only a peer known to have it; notfound frees a sweep request

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* tests: four compact peers are each asked for the block by inv, not by the sweep

The tip-following sweep now asks one peer per block, so it no longer asks
every peer for the same block.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6a18e0f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant