fix: stop NoiseRateLimiter retaining every peer it has ever seen - #1670
Open
Chessing234 wants to merge 1 commit into
Open
fix: stop NoiseRateLimiter retaining every peer it has ever seen#1670Chessing234 wants to merge 1 commit into
Chessing234 wants to merge 1 commit into
Conversation
handshakeTimestamps and messageTimestamps were only ever filtered for the peer being queried. A peer that handshaked or messaged once and never came back kept its dictionary entry for the lifetime of the process, so both maps grew with every distinct PeerID the node had ever admitted -- and PeerIDs churn as geohash peers come and go and as identities rotate. reset(for:) is the per-peer counterpart and is never called from production code (only from its own test), so nothing reclaimed them. resetAll() only runs from clearEphemeralStateForPanic(). Sweep peers whose timestamps have all aged out of their own window, at most once a second so the message admission path does not take an O(peers) walk up to maxGlobalMessagesPerSecond times a second. Both maps now hold only peers seen inside their window. Also inject the clock. Every sibling limiter takes `now` as a parameter -- SyncResponseRateLimiter, BLESubscriptionAnnounceLimiter, BLEAnnounceThrottle -- while this one read Date() inline, which is why none of its time-dependent behaviour had coverage. The default argument keeps all nine call sites unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by reading
NoiseRateLimiter, not from an issue.The bug
allowHandshake(from:)andallowMessage(from:)filter stale timestamps forthe peer being queried:
Nothing filters the dictionaries themselves. A peer that handshakes or messages
once and never returns keeps its entry — with its now-meaningless timestamps —
for the lifetime of the process, because the only code that would drop it is the
filter that runs when that same peer is queried again.
So both maps grow with every distinct
PeerIDthe node has ever admitted. Thatis not a bounded set: geohash peers churn, and identities rotate.
Two things confirm nothing else reclaims them:
reset(for:)is the per-peer counterpart and is never called fromproduction code — the only reference is its own test,
test_reset_clearsPerPeerHandshakeLimit. Peer cleanup was evidently intendedand never wired up.
resetAll()runs only fromclearEphemeralStateForPanic().It is also the component least able to afford this: an attacker can add an entry
per spoofed
PeerIDat the cost of one admitted handshake each, against the verytype that exists to bound handshake cost.
For contrast, the sibling
SyncResponseRateLimiteralready hasprune(now:)— "Drops history outside the window so departed peers don'taccumulate". This type is the outlier.
The fix
Sweep peers whose timestamps have all aged out of their own window (60 s for
handshakes, 1 s for messages), so each map holds only peers seen inside that
window.
Rate-limited at once per second. Sweeping on every admission would put an
O(peers) walk on the message path, which runs up to
maxGlobalMessagesPerSecondtimes a second; once a second is frequent enoughgiven the windows being enforced are 1 s and 60 s.
Also injects the clock. Every sibling limiter takes
nowas a parameter —SyncResponseRateLimiter.shouldRespond(to:now:),BLESubscriptionAnnounceLimiter.decision(for:now:),BLEAnnounceThrottle.shouldSend(force:now:)— while this one calledDate()inline. That is precisely why none of its time-dependent behaviour had coverage,
and why this went unnoticed. The default argument (
= Date.init) leaves all ninecall sites untouched.
trackedPeerCountmirrorsBLESubscriptionAnnounceLimiter.trackedCentralCountso retention is observable from a test.
Test plan
swift test— 213 XCTest cases and 2018 swift-testing cases, 0 failures.Baseline on
main: 210 XCTest and 2018 swift-testing, also green. The deltais exactly the 3 tests added here.
against unmodified
main:XCTAssertEqual failed: ("21") is not equal to ("1")for handshakes and("11") is not equal to ("1")for messages — i.e. every departed peer wasstill held.
pruneStalePeersLocked(now:)calls and re-running: the same two tests failand nothing else does.
New coverage:
peers, then one query past 60 s; only the live peer should remain.
1 s window.
has exhausted its per-peer budget must stay rate limited across a prune,
so the sweep cannot be "simplified" into something that resets budgets and
makes the limit bypassable by waiting one second.
The five existing tests are untouched and still pass.
Not in scope
reset(for:)is still uncalled. Wiring it to peer departure would reclaimentries sooner, but it needs a disconnect hook and is a separate change; the
sweep here bounds the maps regardless of whether that ever lands.