Description
When two Lodestar nodes are configured to peer exclusively via --directPeers (with --discv5=false), the connection is established but immediately torn down during the status/metadata handshake. This prevents isolated two-node setups from syncing.
Error Sequence
- Node B connects to Node A (inbound connection established)
- Node A attempts status exchange:
Req error method=status, version=2 ... Too many messages for missing streams
- Node A marks the request as invalid:
invalid requestStatus peer=<B>
- Node A disconnects Node B with a reconnection cool-down:
onLibp2pPeerDisconnect ... coolDownMin=-1
- Node B range-sync batches fail with
REQUEST_ERROR_DIAL_ERROR / the connection is being closed
Reproduction
# Node A (server)
lodestar beacon --directPeers /ip4/127.0.0.1/tcp/9201/p2p/<B_PEER_ID> \
--targetPeers 1 --disablePeerScoring --supernode ...
# Node B (syncer)
lodestar beacon --discv5=false \
--directPeers /ip4/127.0.0.1/tcp/9300/p2p/<A_PEER_ID> \
--targetPeers 1 --disablePeerScoring ...
Node B stays at connected=0 and cannot sync.
Environment
Analysis
The Too many messages for missing streams error originates from the yamux stream multiplexer when multiple req/resp protocols (status, metadata, ping) race during the initial handshake. The rapid exchange creates messages for stream IDs that have not yet been fully opened, causing yamux to reject them.
This only manifests in small peer sets where the same peer is the target of multiple concurrent protocol requests immediately after connection. In larger networks with gradual peer discovery, the handshake completes before the next protocol request fires.
Impact
- Blocks isolated Lodestar-to-Lodestar e2e testing setups
- Does not affect normal devnet/mainnet operation (peer discovery spreads connections over time)
- Workaround: use mixed-peer discovery instead of strict direct-peer isolation
Related
Discovered during e2e testing of PR #8985 and PR #8995 on epbs-devnet-0.
Description
When two Lodestar nodes are configured to peer exclusively via
--directPeers(with--discv5=false), the connection is established but immediately torn down during the status/metadata handshake. This prevents isolated two-node setups from syncing.Error Sequence
Req error method=status, version=2 ... Too many messages for missing streamsinvalid requestStatus peer=<B>onLibp2pPeerDisconnect ... coolDownMin=-1REQUEST_ERROR_DIAL_ERROR/the connection is being closedReproduction
Node B stays at
connected=0and cannot sync.Environment
test/epbs-e2e-combined(epbs-devnet-0 base + PR feat: add execution_payload_envelopes_by_range serving #8985 + PR fix(sync): harden range req/resp retries and envelope completeness #8995)Analysis
The
Too many messages for missing streamserror originates from the yamux stream multiplexer when multiple req/resp protocols (status, metadata, ping) race during the initial handshake. The rapid exchange creates messages for stream IDs that have not yet been fully opened, causing yamux to reject them.This only manifests in small peer sets where the same peer is the target of multiple concurrent protocol requests immediately after connection. In larger networks with gradual peer discovery, the handshake completes before the next protocol request fires.
Impact
Related
Discovered during e2e testing of PR #8985 and PR #8995 on
epbs-devnet-0.