leios testnet

DeltaQ report validation and driving Leios parameters

Testnet trace collection

Leveraging DeltaQ tracing mechanism (including the proposed TraceRecvBurstSDU):

Aggregated TraceRecvDeltaQSample events (one per peer connection per ~10 s — protocol-mixed today, since the per-protocol split is the unimplemented Shape B in track.md; estBurstS rides on this same mixed sample): a ~100-node testnet with ~60 peers each gives you ~600 samples/sec cluster-wide, ~8 GB/day at ~150 B/sample (see the Shape-B projection under "Production cost" for the per-protocol multiplier). Shippable to Timescale/Influx or S3+Parquet; Prometheus would strain at this cardinality.
Raw TraceRecvDeltaQObservation and TraceRecvBurstSDU events are far higher volume (per-SDU). Ship them selectively — only enable for a subset of nodes, or during measurement windows — and land them in S3 as Parquet for offline analysis. This is the pattern the mempool-measurements dataset already uses.
PeerRTT.quantile is a live STM reader — you can either poll it periodically at whatever cadence you want, or (better) emit it as a periodic aggregate on the same 10 s cycle.
TraceTCPInfo is already wired in the mux bearer, but conditionally: it requires Linux and the tracetcpinfo cabal flag on network-mux at compile time (default off; flag description flags it as "costly"). When enabled, per-SDU it emits kernel state (tcpi_retrans, tcpi_lost, tcpi_snd_cwnd, tcpi_advmss, tcpi_rcv_space) — the primary loss-rate and rwnd answer for the validation frame. In practice: compile a dedicated tracetcpinfo build for the Linux measurement nodes only, run the vanilla build everywhere else. Treat the resulting numbers as testnet / design-phase telemetry only, not runtime signals — the flag is off in production builds by design (see leios-overlap.md's Frame 1 subsection on TCPInfo for the design-phase framing). Additional motivation: tcpi_rtt cross-checked against the branch's PeerRTT.quantile (application layer) is one of the RTT-attribution cross-checks in improved-peer-ranking.md — a peer whose application-layer RTT is much larger than the kernel's TCP-layer RTT is a signature of application-level adversarial delay. For a BBR-vs-CUBIC completion-time experiment, extend the same (already-wired) TraceTCPInfo with tcpi_delivery_rate — on a BBR socket this is the kernel's bottleneck-bandwidth estimate, i.e. a direct measurement of the estBurstS ceiling — and record the per-connection controller (getsockopt(TCP_CONGESTION)) as a trace dimension. The existing tcpi_retrans/tcpi_lost already supply the loss covariate; the CC tag is what makes an outcome attributable — without it, "BBR didn't help" is indistinguishable from "there was no loss for it to help with". The outcome instrument itself (the per-EB completion-time CDF) needs no change: run a BBR node subset against a CUBIC subset on matched paths and compare.

The mux/DeltaQ traces above are the transport layer. The campaign's headline measurements also need two application-layer logs (from cardano-tracer, not the mux) — listed here for a single inventory, with detail in the linked docs:

Per-EB arrival log — (node, EB_id, arrival_ts) at every measurement node, joined with per-EB certification outcome. This is the cross-node per-EB arrival correlation that leios-priority-measurement.md argues is the #1 measurement to deploy (it yields $F_{\text{full}\mid C}$ plus ~a dozen by-products). cardano-tracer already emits block-arrival events; the EB analog is a small extension. Optionally enriched per EB with a closure-complete timestamp (closure coverage) and arrived_from_peer (path reconstruction) — see that doc.
Per-vote arrival log — (election_id, voter, node, ts) from LeiosNotify/LeiosFetch, for the vote-diffusion half of the certification budget (see Vote diffusion, head-of-line blocking, and Praos priority).

So the infrastructure question reduces to "hook initDeltaQTracer' up to cardano-tracer and add a Grafana dashboard." References to Frame's target the leios-overlap.md report, which discusses expanded ouroboros-network tracing machinery to validate concerns in the DeltaQ report and which in turn is used to analyze Leios itself.

Validation vs. parameter-driving — different use cases

Validation needs known ground-truth inputs and measured outputs. For eg. to validate the Mathis relation in the deltaq report, measure estBurstS and p, which requires either the TraceTCPInfo kernel telemetry already emitted by the mux (Linux + tracetcpinfo cabal flag; see the trace-collection section above and leios-overlap.md's Frame 1 subsection on TCPInfo) or the S-residual probe (a noisy secondary derivation). leios-overlap.md recommends both a controlled tc netem microbench (where you dial p and RTT explicitly) and a testnet campaign for different purposes:

Microbench: does the model form (Mathis or CUBIC) match observed throughput at known p, validating the equation.
Testnet: what is the empirical distribution of estBurstS, RTT quantiles, per-protocol tx-arrival latency across a realistic cross-continental mesh? Feeds the inputs to the equation and tells you whether the CIP-0164 assumed values are conservative or optimistic.

Parameter driving (what should CIP-0164 say?) is exactly what a testnet gives you well. Concretely, the observations feed into the following levers, grouped by the same three tiers used in leios-overlap.md's Frame 1 subsection:

Leios parameter	Data you'd use	Tier
`S_EB_tx` maximum	Largest closure whose measured completion-time p95 meets the 7 s voter deadline — sweep size directly; `estBurstS`×size only to interpolate between sampled sizes	Protocol parameter
`L_vote` (currently 4 s, tunable component of the 7 s voter deadline)	p95/p99 of voter validate CDF — 3·L_hdr + L_vote + measured closure-fetch completion (not `estBurstS`·closure_size, which misses the RTO tail that sets the percentile) + µ_eff·N_txs	Protocol parameter
`L_diff` (certRB diffusion budget, currently 7 s)	p95/p99 of measured blended-multi-hop certRB (8 kB) diffusion completion time (not `estBurstS`×8 kB)	Protocol parameter
Round length (`3·L_hdr + L_vote + L_diff`, currently 14 s)	Sum of the two above, plus header diffusion time	Protocol parameter
`p` (loss-rate assumption in `report.md` §4.2, currently 10⁻⁴)	`TraceTCPInfo` `tcpi_retrans` / `tcpi_lost` distribution across connections	Rationale-doc assumption
Mathis-vs-CUBIC choice (`report.md:1046-1173`)	`TraceTCPInfo` `tcpi_snd_cwnd` trace picks the actual growth law	Rationale-doc assumption
`long_hop_owd` (currently 134 ms)	Empirical p95 of `PeerRTT` binned by peer geography	Rationale-doc assumption
rwnd-vs-cwnd binding (§5.4 caveat 1)	`min(tcpi_snd_cwnd, tcpi_rcv_space)` reveals which is the actual limit	Rationale-doc assumption
BlockFetch per-peer byte watermarks (`calculatePeerFetchInFlightLimits`)	Empirical `PeerGSV` from the branch's window; cross-check via `tcpi_delivery_rate × tcpi_rtt` on Linux (field not yet in the `TCPInfo/Linux.hsc` binding — small addition)	Node-implementation constant
SPO tuning guidance (MSS, IW10, congestion-control choice)	`TraceTCPInfo` per-connection deltas from expected defaults	Node-implementation constant (ops guidance, not spec)

TCPInfo-fed rows are Linux-only, testnet / design-phase only (see the trace-collection section above). All other rows are portable and can be collected on any mux bearer.

For the runtime-adaptive lever the branch also enables (a voter refusing to vote when its local mesh health drops below CIP-0164's assumed budget), see the reader-view section below — it belongs in Frame 2, not in this design-time parameter-driving table.

What a good testnet campaign looks like

For the Leios use case specifically:

Topology: 30–100 nodes across ap-northeast-1, eu-central-1, us-east-2 (mirroring the mempool-measurements setup so π₁ and network data live in the same dataset), plus 5–10 "adversarial edge" nodes on non-AWS or consumer-grade uplinks to probe §7 limitation 6 (South America ↔ APAC, deep-buffer bottleneck links). A pure-AWS mesh misses the tail (higher loss, lower bandwidth, etc.) that actually determines feasibility, ie. successful Leios rounds.
Baseline traffic: run Praos with mainnet-scale block production, plus synthetic tx-submission load matching mainnet mempool arrival rates. The Leios pipeline itself may or may not be running; if it is, you're measuring end-to-end; if it isn't, you're measuring the underlying transport under representative load.
Instrumentation: TraceRecvDeltaQSample on every peer for aggregates. Raw TraceRecvDeltaQObservation on 3–5 designated "measurement nodes" (one per region) for high-resolution offline analysis.
Duration: a week, per §5 of the mempool-measurements dataset. Any shorter and you miss diurnal loss patterns — the day-night cycle of network usage, where packet loss on the same path can swing by orders of magnitude between low-traffic hours (~1e-5) and evening peak hours (~1e-3). A short measurement biases the observed p toward whichever part of the cycle you sampled; a week captures the full cycle plus weekend-vs-weekday variation, giving stable tail estimates. Directly plottable from tcpi_retrans against local time-of-day.
Controlled perturbations: on a sub-mesh, inject tc netem shaping to force known p, RTT, and loss patterns. This gives you the ground-truth pairs needed to falsify Mathis vs CUBIC (i.e., Experiment 1 from leios-overlap.md, run inside the testnet rather than on a two-node isolated rig).
LeiosFetch-scheme validation (if a candidate scheme is deployed on the testnet). Compare the four configurations from leios-fetch-scheme.md's suggested experimental design: memo's scheme as written (baseline), memo + Improvement #1 (hybrid classification only), memo + Improvements #1–#8 (full adaptive variant with fusion and diversity-aware churn), and the erasure-coded variant (Alternative B, if prototyped). Metrics per configuration: empirical $F_{\text{full}\mid C}(14)$, P99 completion time, redundant egress per round, and adversarial resistance under the injection scenarios in item 7 below.
Adversarial injection: designated adversarial nodes on the sub-mesh with controlled attack profiles — tc netem for the transport-level ones, a little adversarial serving logic for the behavioural ones. Tests defensive mechanisms that would otherwise be untested until real-world adversaries appear — several of them the delivery-gaming and ranking-system attacks catalogued in improved-peer-ranking.md:
- Slow-loris: adversarial node accepts fetch requests then delays response indefinitely — no attempt to hide, so it looks slow. Tests Improvement #3's throughput-collapse detection (estBurstS rising) and the ranking (it scores low).
- RTT inflation: adversarial node adds artificial delay to all responses. Tests the RTT-attribution cross-checks (PeerRTT.quantile vs tcpi_rtt divergence, geographic minimum RTT), both covered in improved-peer-ranking.md.
- Pacing (idle-gap): times its SDUs so estBurstS reads fast — a high-bandwidth look — while the closure actually arrives slowly, the mirror image of slow-loris. Tests the promptness-gated burst window, coverage shrinkage (short bursts starve coverage, pulling estBurstS back to the average), and the actual-vs-predicted backstop.
- Byte-padding: serves other traffic (or other mini-protocols) fast while dribbling the closure we asked for, so its aggregate estBurstS stays flattering. Tests per-fetch completion timing (T_wait) and the actual-vs-predicted check on persistence — the signals that isolate our specific fetch from the peer's aggregate.
- Reputation farming → surgical withholding (multi-round): behaves well over low-stakes rounds to earn a top rank cheaply, then withholds the one deadline-critical fetch. Tests the score-based churn (worst slice per hour + random resample) and whether a single well-timed defection escapes it — the peer-side analog of the producer-side Targeted release (T22) below.
- Baseline poisoning (multi-round): inflates its own residuals to widen the tolerance band and hide under it. Tests that the actual-vs-predicted band is drawn from the honest-peer population, not the suspect's own samples.
- Chunk-hedge slow-loris: adversary is one of many NBL peers each getting a chunk; it slow-lorises its assigned chunk. Directly tests the memo's key adversarial concern about block-completion under partial-peer failure.
- Targeted release to voters (T22): an adversarial producer serves the EB body/closure only to a configured subset of committee voters — just enough to certify — and to no other node. Certification succeeds while the rest of the mesh has received nothing from the producer; $\hat{F}$ restricted to the non-voter nodes is then a direct empirical $G_{\text{adv}}$ (see leios-conditional-diffusion.md's adversarial-diffusion section). Tests the CIP's $L_{\text{diff}}$ backstop: whether the >25% honest holders can re-diffuse 12 MB to everyone within 7 s.
- Topology-induced peer concentration (non-adversarial but shares the measurement path): configure a sub-mesh with a stake-concentrated "dense" region and a sparse region with only 2–3 high-stake SPOs. Measure whether churn drives mesh-wide big-ledger peer sets toward the dense region (as predicted by the per-block-origin model in leios-fetch-scheme.md's Improvement #8) and whether sparse-origin blocks then see low mesh-wide seed-hop coverage. Compare against a diversity-aware-churn deployment to quantify how much the mitigation buys.
Sub-mesh scale for adversarial injection should mirror the real-world adversary size assumption (e.g., ~4% of the big-ledger pool, per the arithmetic in leios-fetch-scheme.md's seed-node section).

Where this connects to work already in flight

The mempool-measurements dataset that fed pi1_derivation.md is already collecting some of what you'd need — per-node tx-arrival timestamps. Extending that pipeline to also collect this branch's DeltaQ traces gives you π₁ and the network side from the same nodes in the same time window, which lets you cross-validate: does a period of high π₁ correlate with degraded estBurstS (i.e., is mempool divergence caused by network stress)? That's an interesting question the report can't answer today.

Similarly, the IOG benchmarking cluster that runs network-benchmark topologies is a natural fit — it's already designed to run cross-continental Cardano nodes under controlled load.

Bottom line, and the rest of this document

A real cross-continental testnet is the right validation vehicle, especially for parameter-driving Leios. The traces are already collectible via the existing cardano-tracer infrastructure. Combining testnet measurement with a controlled tc netem microbench gives you both the inputs (empirical RTT/S distributions on real paths) and the falsification (does Mathis or CUBIC fit at known p) that the report currently lacks. The main gaps are (a) adversarial-path coverage — pure-AWS meshes under-represent the tail, and (b) getting the branch's per-protocol transport signals — estBurstS and matched RTT — onto the deployed nodes for a direct reading of tx-submission transport health (the 1-hop pre-diffusion test's transport axis; without them it is an external-consumer reconstruction, per leios-main-vs-branch.md). Neither is a fundamental obstacle.

That covers the validation-and-parameter-driving frame. The rest of this document covers three topics that don't fit inside that frame:

Production cost of the DeltaQ machinery — running the branch's RTT/DeltaQ machinery on public relays (with hundreds of inbound peers) has real cost. What that cost is, what mitigations exist, and how they interact with the two Leios frames.
Adaptive Leios via runtime measurement — the branch enables Leios mini-protocols to observe their own network conditions at runtime and adapt behaviour, rather than baking assumed conditions into CIP-0164 forever. This is the reader-view expansion below.
Fully-blended diffusion when 1-hop fails — what happens, how to diagnose it, and how DeltaQ traces expose dynamics the report itself doesn't model. Covered further down.

Production cost of the DeltaQ machinery

Existing Cardano practice keeps DeltaQ analysis strictly on the trace path (rather than in-band with protocol logic) precisely because it's potentially expensive, and BlockFetch derives its PeerGSV from KeepAlive-driven samples (once every ~10 s) rather than a richer signal for the same reason. This part covers what the branch adds on top of that existing cost, when it matters, and how to mitigate.

What the branch adds on top of main's existing cost

Main already emits TraceRecvDeltaQObservation per SDU and runs TraceStats.step on every observation. That cost is proportional to SDU rate across all connections. The branch layers on:

Cookie mint (egress). PRNG draw + OrdPSQ insert per SDU, amortised by 1 ms mint-batching to roughly "one cookie per outbound burst" rather than per SDU.
Cookie match (ingress). PSQ.atMostView per received SDU: matched-echo lookup + age-out of expired cookies in one operation. O(log n) where n = outstanding sends.
Per-protocol StatsA lookup (only if Shape B is implemented — it is not today; the current aggregator keeps a single protocol-mixed StatsA). Map MiniProtocolNum around the existing per-size record structure. Note what this does and does not cost (see the Shape-B projection below): the hot path barely moves (one Map.adjust over ≤7 keys per observation, negligible against the cookie PSQ work); what scales ~5× is resident state and per-period emit work, not per-observation CPU. Not part of the branch's current cost.
Burst tracker. Single-slot cursor per connection, essentially free.
PeerRTT t-digest update. O(log δ) per matched echo, δ = 100.
TVar write for the reader view: one STM commit per matched echo.

The per-SDU addition is roughly a doubling of main's existing DeltaQ work. Storage per connection: a few kB for the t-digest, a few kB for the (single, protocol-mixed) StatsA state, plus the cookie PSQ (capped at a few thousand outstanding — see the memory note below).

Rough arithmetic for a heavily-loaded public relay (500+ connections, ~10 k SDUs/s aggregate, ~5–10 µs of DeltaQ work per SDU): 5–15% of a core (unbenchmarked estimate — flagged for measurement) and cookie PSQ storage of ~100–250 MB worst-case at the 4096-cap (≈ 4096 entries × ~60–120 B boxed × 500 connections; per-entry size and CPU cost are estimates, pending benchmarking). Not catastrophic in isolation, but additive with everything else on the node — and the kind of overhead current Cardano practice explicitly refuses to pay in the block-fetch decision path.

If Shape B (per-protocol DeltaQ) were implemented

The cost model above is for the current single-StatsA, protocol-mixed aggregator. Shape B (track.md) buckets by MiniProtocolNum, emitting one sample per active protocol per connection per period instead of one per connection. Its cost multiplies by the active-protocol count (≤5 upper bound; 2–4 in practice, since not every hot protocol carries traffic on every connection in a 10 s window):

Quantity	Current (single)	Shape B (≤5×)
Sample rate, cluster (100 nodes × ~60 peers)	~600/s	~3,000/s
Aggregated-trace volume, cluster (@~150 B)	~8 GB/day	~40 GB/day
Resident StatsA state, per connection	~1–3 kB	~5–15 kB
StatsA state, per 500-conn relay	~1.5 MB	~7.5 MB
Per-observation (hot-path) CPU	baseline	~unchanged

The hot path barely moves: one extra Map.adjust over ≤7 keys per observation, negligible against the cookie PSQ's per-SDU atMostView. What scales ~5× is resident state and the 0.1 Hz per-period emit work (5 regressions + 5 samples instead of 1) — so Shape B leaves the "5–15% of a core" ballpark unchanged and multiplies sample count, trace volume, and state instead. On the bounded measurement nodes (~30–50 connections) it is trivial either way; the ~40 GB/day aggregated cluster volume is shippable to Timescale/Influx or S3+Parquet (Prometheus would strain at the per-connection-per- protocol cardinality), and is dwarfed by the raw TraceRecvDeltaQObservation / TraceRecvBurstSDU streams the campaign already ships selectively.

The tx-submission direction inversion

Almost every Cardano mini-protocol carries data from responder to initiator — block-fetch, chain-sync, keep-alive. Tx-submission is inverted: the initiator sends transactions to the responder. See track.md's cross-cutting note on this for the general framing.

Practical consequence for cost mitigation: from a relay's perspective, the tx-submission traffic we care about for pre-diffusion monitoring arrives on inbound connections — where this node is the responder, and where peer count is highest (tens to hundreds). Blocks arrive on outbound connections, which is the low-peer-count side.

Any mitigation that turns off DeltaQ machinery "on inbound because no Leios logic reads it" is therefore wrong — it silences the exact tx-submission signal Frame 2's pre-diffusion diagnostic depends on (see The failure-mode diagnostic table). Direction-based gating is not a safe cost-reduction strategy.

Mitigation strategies

Ordered by leverage. Choose a combination based on which frame you're supporting and how much per-peer fidelity the consumer actually needs.

1. Statistical sampling across inbound peers.

Frame 2's pre-diffusion diagnostic asks a mesh-aggregate question — "is gossip healthy across my peers?" — not a per-peer one. Instrument a random 30–50 of 500 inbound peers rather than all 500. Statistically indistinguishable for aggregate purposes at ~10x lower cost. Rotate the sample every few minutes to avoid systematically favouring a subset.

Does not help if the consumer needs per-peer signals, but for the pre-diffusion diagnostic specifically it's a large, safe reduction.

2. Protocol-scoped enablement.

Only run cookie/burst machinery for mini-protocols whose signals are consumed:

Tx-submission on inbound → for pre-diffusion monitoring.
Block-fetch on outbound → for peer selection.
Skip chain-sync (no consumer for its RTT signal here) and keep-alive (already provides a lightweight RTT signal, doesn't need cookies).

Roughly halves per-connection state and per-SDU cost by trimming the per-MiniProtocolNum map to just the protocols in use.

3. Probe sampling per SDU.

Cookie 1-in-N SDUs rather than every SDU. Reduces PSQ churn linearly. RTT is stationary enough that 1-in-4 or 1-in-10 sampling still gives hundreds of samples per minute per peer — well above KeepAlive's 6-per-minute baseline.

4. Aggregate-only mode.

For consumers that only ever ask "median across my peers", collapse the per-peer RTT windows into a single shared window per protocol. Trade per-peer visibility for cost. Not compatible with peer-selection use cases (which need per-peer numbers) but fits Frame 2 pre-diffusion monitoring cleanly.

5. Compile-time flag.

Gate the whole Network.Mux.RTT module behind a deltaq_rtt cabal flag, mirroring the existing tracetcpinfo pattern. Off by default. Ultimate opt-out for operators unwilling to accept any DeltaQ cost.

Downstream consequence: a node with the flag off cannot participate in any Frame-2 behaviour. This bifurcates the network into "Frame-2-capable" and "vanilla" nodes — a real deployment concern requiring governance consideration, not just an ops decision.

6. Consumer-side aggregation.

Move StatsA aggregation off the mux path; ship raw TraceRecvDeltaQObservation and TraceRecvBurstSDU events to cardano-tracer and aggregate at the consumer. Trades in-mux CPU for trace-forwarding bandwidth. Practical for testnet measurement nodes where the consumer is close by; impractical for production nodes where raw per-SDU trace volume would swamp the forwarder.

What this means for the two Leios frames

Frame 1 (validation / parameter driving) runs on ~30–50 instrumented measurement nodes with a bounded connection count. Full machinery is affordable at that scale — no mitigation needed for the campaign itself.

Frame 2 (adaptive Leios in production) requires the machinery to run on public relays with hundreds of inbound connections. A defensible production configuration:

Statistical sampling across inbound peers (~30–50 of ~500), with rotation.
Protocol-scoped enablement (tx-submission on inbound, block-fetch on outbound).
Probe sampling at 1-in-4 or 1-in-10.
Compile-time flag available as the final opt-out for operators who refuse the cost.

The compile-time flag creates a network bifurcation. Nodes with the flag off can't participate in Frame-2 behaviours (voter self-throttle, adaptive S_EB_tx, runtime DeltaQ report's Rec 3 enforcement). Whether Frame-2 behaviours are advisory (nodes MAY adapt) or mandatory (nodes MUST adapt) is a CIP-0164 governance question the branch doesn't answer.

The mitigation-vs-safety interaction

Frame 2 changes the shape of the network's protocol behaviour in a way Frame 1 does not. In Frame 1 the branch only measures — nodes either produce more or less data, but the protocol is unchanged. In Frame 2, nodes voluntarily deviate (throttle, opt out) based on local observation. That deviation is only sound if the deviation criteria are so rare that variation doesn't move P_cert, or if every node's deviation happens under sufficiently correlated conditions.

The cost mitigations above interact with this. Statistical sampling introduces variance in what each node observes, which introduces variance in when each node throttles. If observation is uncorrelated across nodes, the network gets graceful degradation. If observation is correlated (all nodes throttle at the same moment because they observe the same shock), the mesh loses quorum en masse and P_cert collapses. This is the cross-node covariance question from the blended-diffusion section, applied at the level of adaptive decisions rather than pre-diffusion state.

So cost mitigation isn't just an engineering optimisation — it interacts with the operational safety of the Frame-2 lever. That interaction is worth measuring on the testnet before committing to any particular mitigation combination in production.

Measurement validity under load

The whole campaign — and Frame 2 especially — trusts the branch's own telemetry. But that telemetry shares a runtime with the workload it measures, and several signals degrade precisely during the heavy-round windows that matter most.

Self-inflicted bufferbloat in PeerRTT. RTT cookies are stamped at SDU construction, before the blocking socket write (the Egress.hs batching comment notes stamping otherwise inflates the samples). So while a node is doing its own bulk egress, its outbound cookies sit behind megabytes of its own queued closure bytes, and the RTT it measures to a peer is inflated by its own send-buffer occupancy, not the path. TCP_NOTSENT_LOWAT (Linux/Darwin) bounds this by capping the not-yet-sent backlog (gap 17's stack); without it, a node's PeerRTT is least trustworthy exactly when it is busiest.
Echo timing is workload-shaped. A cookie echo rides the peer's next outbound SDU; on a quiet reverse path the earliest echo is the next KeepAlive fire (~10 s away). So both the sample rate and the measured value depend on reverse-direction traffic, not just the path.
GC pauses land in the samples. A multi-second GC pause during 12 MB deserialisation/validation is charged to whatever cookie or burst gap straddles it — inflating RTT and depressing estBurstS during the busiest rounds.
Sample rate covaries with load. More traffic → more cookies → more samples; quiet peers contribute few. Cross-peer quantile comparisons inherit the bias.
Mixed-version selection bias. The cookie fields reinterpret SDU header bytes (see track.md); a peer on old code never echoes, so in a partially-upgraded mesh telemetry comes only from upgraded peers — which skew toward better-run operators. That biases the very campaign meant to set CIP parameters, and the wire-compatibility / version-negotiation story for the cookie fields is not yet written down.

The sharp consequence for Frame 2. This is why the adaptive decisions key on the estimate and on observed fetch progress, not on a raw PeerRTT.quantile compared to an assumed p95 — which would compare unlike quantities: the observed value embeds the node's own bufferbloat, peer scheduling, and GC, inflated exactly in the stressed windows where the decision fires. Observed progress sidesteps most of these artifacts — the inbound payload arrives at its real rate whatever the node's outbound send-buffer is doing, and a synchronised GC pause is caught by the cross-peer common-mode reject. Whatever thresholds remain must be calibrated against the observed distribution under load, not the path assumption; the RTT cross-checks (PeerRTT vs tcpi_rtt, geographic-minimum RTT) stay for attribution, per the companion improved-peer-ranking.md.

A cheap first-line lever is quantile selection. The no-echo and GC-pause confounds are sparse and pile up at the extreme top of the distribution, so reading a somewhat lower quantile (say p85–p90 rather than p99) sits below where those artifacts dominate while staying tail-sensitive enough to flag real degradation — a deliberate trade of a little extreme-tail sensitivity for robustness against the contributions we want to exclude. Since PeerRTT.quantile takes the quantile at read time, the consumer picks that operating point directly. This helps the sparse-outlier confounds only: sustained self-bufferbloat during bulk egress shifts a broader part of the distribution (needs TCP_NOTSENT_LOWAT), and the mixed-version bias is about which peers contribute, not one peer's tail.

On the testnet, quantify the bias directly: compare a node's self-measured PeerRTT against external ping / kernel tcpi_rtt during its own bulk-egress windows versus idle.

Adaptive Leios via runtime measurement

What the reader view actually enables

Static vs. adaptive parameter setting — the category shift

The Leios report and CIP-0164 both operate in what one might call the static-analytical mode: pick network conditions (RTT distribution, congestion model, loss rate), solve for parameters (L_vote, L_diff, S_EB_tx) such that P_cert ≥ threshold, freeze those parameters into the protocol. Once frozen, every node applies them identically, blind to whether the actual mesh conditions at that moment resemble the assumed ones. A voter that cannot fetch its closure in time still votes on the same deadline as one that can; a block producer still packs closures to the same S_EB_tx regardless of what its actual peers can absorb.

The reader view breaks this: a live Leios voter (or block producer, or non-voter fetcher) can read, at runtime, not merely the RTT to each peer but the full time-to-deliver estimate ramp(RTT) + estBurstS × size for a transfer over that link (the companion improved-peer-ranking.md), refined mid-fetch by the observed progress of the transfer in flight. That is the category shift from offline-analytical to online closed-loop. We key these decisions on the estimate, not raw RTT; the companion explains why.

Concrete adaptive behaviours it enables

Ordered roughly by conservatism:

Self-diagnosis / operator alerting. The mildest use. Compare the node's measured time-to-deliver for a typical closure — its estimate to representative peers, or the observed completion of recent fetches — against the budget CIP-0164 assumed. If it exceeds the envelope by some margin, log a warning. Doesn't change protocol behaviour; just tells operators their node is operating outside the CIP-0164 design envelope.
Adaptive peer preference. BlockFetch already selects peers by PeerGSV, but its update path is via KeepAlive's ~10 s cadence — coarse. The window-based signal from this branch is per-connection, updated on every response, and covers all miniprotocols. Peer-selection could react seconds faster — ranking on the estimate, and on measured T_wait where fetches have completed, which is what the companion builds to be adversarially robust, rather than on raw RTT drift.
Adaptive S_EB_tx sizing at production time. A block producer preparing an EB can invert the estimate: from its measured link conditions to typical peers, pick the largest closure size S whose predicted diffusion time ramp(RTT) + estBurstS × S still fits the certification budget, and cap below S_EB_tx_max accordingly — voluntarily accepting a smaller EB to keep certification probability up. Global CIP-0164 sets a ceiling; the node sets the actual size per-round from its measured delivery capacity. This is also the mesh's stabilizing feedback against sustained-overload livelock — see the Cascade to livelock section below.
Adaptive Freshest-First Delivery (FFD). FFD is the "when I'm behind on diffusion, prioritise the newest EB and drop the older" mechanism the report defers modelling (§7 limitation 7). Its trigger is fundamentally "am I going to make my deadline?" — time remaining against the estimate for the work remaining. Before a fetch that is ramp(RTT) + estBurstS × work_remaining; once one is in flight it is the live projection from observed progress — this round's measured rate on the bytes still outstanding — which also captures the RTO tail that estBurstS × size misses (the reason the parameter table prefers measured completion). Drop the older EB when that projection says the newest won't otherwise land. Static FFD thresholds are guesses; the reader view makes the projection possible.
Voter self-throttling. The most aggressive use, and the one that most needs the robust signal. The decision is FFD's own "am I going to make my 7 s deadline?" (item 4): project this round's completion from the voter's observed fetch progress — the live rate on the closure actually arriving, against the bytes still unfetched and the time left — and bow out if it won't finish, rather than voting and later failing to diffuse the certRB. Anchoring on observed progress is essential, not incidental: because it is direct measurement, a set of peers inflating their echoed RTT cannot trick the voter into standing down (see Why this doesn't come for free, below). PeerRTT.quantile above the CIP-0164 assumption is at most a coarse pre-fetch prior and a mesh-health cross-check. This trades quorum size for quorum quality — fewer votes, but the ones cast are more likely to diffuse.

The interesting property that runs through 3–5: they let a node voluntarily degrade under stress in ways the static analysis can't. The report's P_cert collapses to ≈ 0 under catastrophic pre-diffusion failure because every voter tries and every voter fails. An adaptive scheme where stressed voters bow out could keep the quorum threshold met among the healthier subset — turning a hard cliff into a graceful degradation.

Detecting mesh-wide stress

Behaviours 3 and 5, and the livelock stabiliser below, all want the same gate: back off only when the mesh is broadly overloaded, not when one or a few peers are struggling — which is the normal background and means nothing. The per-peer machinery already draws that line in one special case: the cross-peer common-mode reject of improved-peer-ranking.md suppresses a stall seen on every peer at once as our-side-or-network, not any peer's fault. Mesh-stress detection generalises it from synchronised stalls to synchronised degradation, and poses it as a belief we update rather than a threshold we trip.

The observation, normalised per peer. Absolute rates can't be compared — a far peer is legitimately slow — so score each peer against its own baseline (the peer-ranking prior μ_p): is its recent delivery (estimate, or measured T_wait) more than a margin below μ_p? The per-tick observation is the count x of such peers out of N active; a large x/N is a correlated dip below per-peer norms — the signature of a mesh event rather than of the ever-present few slow peers. A second, slower channel guards the sustained case, since the adaptive baselines eventually follow a lasting slide and stop firing: the mesh's aggregate delivery against the fixed CIP-0164 envelope (the self-diagnosis comparison above), which cannot adapt away. This two-channel split — relative dip-count for speed, fixed envelope for the frame — is the relative signals, absolute anchors invariant of improved-peer-ranking.md.

The belief, by recursive Bayes. Rather than threshold x/N, carry a posterior P(stressed) over a latent mesh state that evolves slowly (two states with a low switch probability, or a continuous level on a random walk), updated each tick from the observation. This is the dual of the peer-ranking shrinkage — there we borrow the population to sharpen one peer; here we pool the peers to estimate what they share, the common shift — and it inherits three properties a bare index would have to bolt on:

Hysteresis, from the transition prior. A state a priori slow to switch means one noisy tick barely moves the posterior while sustained correlated evidence moves it fast — the damping the adaptive behaviours require, derived rather than dialled in, with no separate band.
Small-N shrinkage, from conjugacy. With x ~ Binomial(N, θ) under a Beta prior on θ, the posterior is Beta and wide when few peers are active, so "2 of 3 down" yields low belief on its own. "Sharpens with peer count" is then just the posterior tightening as N grows, not a fallback.
Reliability-weighting. Weight each peer's evidence by its own precision (coverage, variance), so a well-established peer counts and a thin or new one barely does — the same σ²/τ² decomposition the coverage scale k uses, now on the shared factor. Pool it robustly as well — a median or trimmed aggregate over the per-peer evidence, not a raw mean — so a few outlier or adversarial peers cannot swing the belief: the "median, not worst" guard, applied to the whole distribution.

A second dimension: mempool fragmentation. The same filter generalises past transport. Failing pre-diffusion — the referenced txs not yet in the local mempool when an EB arrives — shows up as an elevated observed π₁ (the missing-tx fraction; Diagnosing pre-diffusion below), and when that is persistent across rounds and broad across EB sources (the mempool systematically behind whoever produced the block, not one front-running producer) it is a second stress tell. Keep it a second latent dimension rather than folding it into transport, because the two are distinct: a healthy pipe can coexist with a fragmented mempool (a gossip pathology, partition, or withholding). The joint belief over {transport-stress, mempool-fragmentation} is then the continuous form of the failure-mode diagnostic table below — its quadrants are the belief's modes, and its transport-vs-π₁ cross is the causal split (elevated π₁ with degraded transport is one cause driving both; with a healthy pipe is the distinct pre-diffusion failure).

The two dimensions drive the same back-off lever through different terms of the estimate, which keeps the coupling clean: transport stress raises the rate term (ramp + estBurstS), while mempool fragmentation raises the size term — a fragmented mempool makes fetchers cache-miss and pull the missing txs, so the effective closure balloons. That is exactly the "failing pre-diffusion → larger closure fetches" effect noted elsewhere, and behaviour 3's size-aware sizing already caps S_EB_tx when size rises, for the same certification-budget reason. One belief, two entry points.

Deciding, and by cause. Act when P(stressed) crosses a point set by the asymmetric loss — a missed overload risks the livelock cascade, a false backoff costs some throughput — not a hand-picked threshold. Separate the cause with a third state, {healthy, network-stressed, self-paused}, where agreement between app-layer RTT and kernel tcpi_rtt is the observation that tells a real path slide from our own GC/scheduler pause: back off production under network stress, where shedding load helps; don't bother under a self-pause, where it doesn't. A parallel split is spatial — cluster the degraded peers by region or path: tight clustering is a regional partition or regional congestion rather than global overload, and calls for routing around the region, not a mesh-wide back-off.

The lever, and the cost. On high belief, cap S_EB_tx below the CIP ceiling (the Cascade to livelock stabiliser), trim voting commitment (behaviour 5), defer non-urgent fetches; release as the belief decays, the transition-prior damping keeping the release from synchronising across the mesh. The filter is O(1) per tick — one conjugate Beta update and one multiply-normalise on the state posterior — so it costs nothing over counting, and reuses the per-peer μ_p, T_wait, and common-mode signals already maintained.

Calibration and validation. The filter's knobs come from the testnet traces, not from feel. The transition prior follows the measured autocorrelation timescale of degradation episodes — how long stress actually persists — so its built-in hysteresis matches reality rather than a guessed horizon.

The decision threshold — how high P(stressed) must climb before we back off — is set by weighing the two mistakes, whose costs are lopsided: missing a real overload risks the livelock cascade, whereas a false backoff merely forfeits some throughput for a round. That imbalance is the asymmetric loss, and because a miss is far dearer than a false alarm it pulls the trip point well below an even 0.5 — the valve fires early on purpose.

That threshold only means something if the belief is calibrated — if its numbers mean what they say. A calibrated P(stressed) = 0.8 means that across all the moments it read 0.8, the mesh really was stressed about 80% of the time; one that reads 0.8 when the truth is nearer 50% is overconfident and would trip wrongly. Two standard tools measure this against the recorded outcomes. A reliability diagram bins the predictions and plots, per bin, the average predicted probability against the observed frequency of stress — perfect calibration lies on the diagonal, and departures show over- or under-confidence. The Brier score condenses the same check into a single number: the mean squared gap between the predicted probability and the 0/1 outcome (lower is better). Where either shows miscalibration, the prior and threshold are adjusted to close it.

Finally, the belief earns its place as a safety valve only if it leads certification failures by enough margin to act on — measured by its cross-correlation with the failure-autocorrelation series (below). A signal that merely coincides with failures is a diagnosis, not a valve.

Traces for offline study

Studying an episode after the fact — measuring the lead, fitting the transition prior, checking calibration — needs the right traces, and the governing principle is to log the belief's inputs, not just P(stressed). A logged belief can only be replayed under the parameters that were live when it was written; the observations behind it can be re-run through the filter under any transition prior or threshold, which is exactly what re-tuning and calibration require. Emit three streams on one monotonic clock, each carrying a slot/round reference for alignment.

The mesh-stress tick — one record per filter update (≈ 1 Hz, faster if episodes are short):

the timestamp and current slot;
the belief P(stressed) and its components P(transport), P(mempool) (and the three-state cause posterior, if run);
the observation that drove the update — x (peers below their own μ_p by the margin) and N (active peers), the robust aggregate value, the aggregate π₁ over recent EBs, the envelope ratio (measured time-to-deliver ÷ CIP-0164 budget), the app-vs-kernel-RTT agreement summary, and the per-region degraded counts;
the filter parameters in effect (transition probability, threshold, margins), so a replay knows the live configuration it is reproducing.

The round outcome — largely the per-EB certification log already proposed as the campaign's #1 measurement (leios-priority-measurement.md), one record per round:

the slot/round id and timestamp (same clock);
certified? / failed? and the failure mode;
the per-EB observed π₁, the closure size actually fetched, and the fetch's T_wait.

This is both the outcome series the belief is cross-correlated against and the source of the failure-autocorrelation series itself.

Valve and action markers — one record per event:

the timestamp, whether the belief crossed the arm or the release threshold, and the action taken (the S_EB_tx cap applied, a voter bow-out, a deferred fetch).

These delimit episodes and carry the follow-on question past lead — whether acting actually helped.

A fourth stream is optional and sampled: a per-peer snapshot (peer id, region, current estimate / T_wait, μ_p, coverage/precision, degraded flag, app-vs-kernel RTT), taken every few seconds or on threshold crossings rather than every tick. It is too heavy for the hot path, but it reconstructs which peers led and the spatial structure when an episode warrants a closer look.

What each analysis then draws on:

Lead — the lag at peak cross-correlation of the P(stressed) series against the failure indicator, segmented per episode by the markers.
Transition prior — the persistence timescale read off the observation series' autocorrelation.
Calibration — the P(stressed) samples binned against the realised "stressed" outcome (defined from the round log: the round failed, or T_wait / π₁ breached a margin), scored as a Brier score or reliability diagram.
Re-tuning — replay the observation records through the filter under candidate parameters; the live node never has to reproduce a configuration.

Why this doesn't come for free

Three problems, in decreasing order of severity:

Adversarial gaming — being tricked into standing down. A colluding set that delays echoing our cookies inflates our locally-observed RTT to them. Against peer demotion this is self-correcting — a peer that echoes slow ranks itself down and we route around it. The dangerous reaction is self-throttling (item 5): if a bow-out keyed on locally-observed RTT, a small colluding set could inflate it and push a targeted voter out of the committee. The primary defence is structural, and it is why item 5 keys on observed progress: base the bow-out on the closure bytes actually arriving, not on RTT. A colluder can delay its echoes but cannot make the payload we are receiving appear absent — a voter on track to finish votes regardless of an inflated RTT. Aggregating across peers (median, not worst) and the cross-peer common-mode reject — a genuine mesh-wide slowdown shows on all peers at once, an attack on a few — are secondary cross-checks.
Feedback loops in the mesh. If some nodes self-throttle and others don't, the ones still voting have to carry more of the diffusion load, potentially degrading the observed RTT for them, which could cascade. This is a classic control-theory pitfall. Any adaptive behavior needs damping — a hysteresis band, or a stochastic component in the throttle decision — to avoid synchronized oscillation across the mesh.
Governance / spec-compliance. CIP-0164 is a consensus-critical spec; adaptive voter behaviour that varies decisions based on local observation is a soft deviation from "every voter runs the same protocol." Whether an adaptive voter's cast/not-cast decision is spec-compliant, or whether the CIP needs to explicitly allow (or mandate) it, is a governance question — not a technical one. The technical prerequisite (the reader view) is there; the political prerequisite isn't.

Why it's specifically interesting for Leios

Praos already has adaptive elements (peer selection, BlockFetch churn, ChainSync candidate ordering) that consume PeerGSV. So the pattern — miniprotocol reads network telemetry, adjusts behavior — is not new to Cardano. What's new is:

Leios's deadlines are tighter. Praos's mean block interval is 20 s (active-slot coefficient 0.05 over 1 s slots), but that is not a per-block diffusion budget — blocks can be one slot apart, and the operative diffusion target is Δ ≈ 5 s. Praos typically delivers within 1–2 s, so ~2–3× headroom against Δ. Leios's 7 s voter deadline against a 4.4 s modelled long-haul fetch has ~1.6×. Still tighter — but the gap is ~2–3× vs ~1.6×, not 10× vs 1.6×, so the "category shift" is real but milder than a naive slot-interval comparison suggests. Variance that Praos absorbed starts to matter.
Certification is atomic. A slow block in Praos still gets adopted eventually; a slow EB simply doesn't certify. There's no "eventually" — either you make the 7 s deadline or you don't. That's exactly the regime where local decision-making beats global static settings.
The report's own recommendation 3 depends on it. "Ensure effective tx-submission pre-diffusion" (report.md:1584-1588) is currently unsatisfiable at protocol design time because pre-diffusion is a runtime property. But it's satisfiable at runtime: a node observing degraded pre-diffusion (via per-mini-protocol DeltaQ traces on tx-submission) can react — refuse to vote, reduce closure size, alert. The PeerRTT reader view (and its per-protocol siblings) is the API that turns Recommendation 3 from unsatisfiable to enforceable.

The short answer

The reader view means Leios doesn't have to bet its safety on the network conditions assumed at CIP-0164 design time being the ones present at runtime. It can measure, and adjust. That's a lever the report's static-analytical framing doesn't consider, and probably the single largest place where the branch's work changes what Leios can do, as distinct from what the report can verify.

Fully blended diffusion

The reader-view section covered adaptive protocol behaviour in the good case, where the mesh is broadly healthy and a node is fine-tuning around CIP-0164's assumed baseline. This part covers the degraded case: when 1-hop pre-diffusion has failed and the full EB closure has to traverse the multi-hop mesh, with the report's α · P_{cert,1-hop} mixture collapsing towards the blended-diffusion tail. Two subsections — one on diagnosing which failure mode you're in, one on investigating the dynamics of the mode itself.

Diagnosing pre-diffusion: transport vs. outcome

The report's Limitation 1 and Recommendation 3 both hinge on pre-diffusion — the assumption that transactions have already reached local mempools via tx-submission gossip before the EB referencing them is produced. Testing whether that holds, and diagnosing why when it doesn't, needs two signals from different layers. Neither is sufficient on its own; conflating them (as the report's α mixture parameter implicitly does at report.md:1584-1588) loses information a production node needs.

The two signals

Transport-layer (mux-layer, from the branch's traces). Per-mini-protocol DeltaQ traces bucketed to tx-submission give, per peer, the time-to-deliver estimate for the tx exchanges — ramp(RTT) + estBurstS × size over RequestTxs → ReplyTxs (and the lighter RequestTxIds → ReplyTxIds) — folding round-trip and throughput into one figure, with a dead channel (the peer not responding at all) the degenerate case. Interpretable as "how quickly can the tx-gossip pipeline actually deliver right now, per peer?" Raw per-exchange RTT is a component of that estimate and a cross-check, not the verdict on its own.

Application-layer (from EB processing, not mux). At each EB arrival, cross-reference the EB's referenced tx-hash list against the local mempool. The fraction not present is the observed π₁ for that EB, at this node, at this moment. Interpretable as "how out of sync was my mempool at the moment I needed it?"

The report's mempool-measurements dataset already collects the outcome signal at coarse granularity (per-node, per-block). The mux traces contribute the transport signal, per-connection, at higher temporal resolution. Both are needed.

The failure-mode diagnostic table

Composing the two — each judged against its per-region-pair baseline (below) — gives a diagnostic that distinguishes qualitatively different failure modes. Transport: Degraded means the tx-delivery estimate (or the measured completion of RequestTxs → ReplyTxs) sits above baseline — round-trip and throughput together, not a raw-RTT threshold; π₁: Elevated means the missing-tx fraction sits above baseline.

Transport health	Observed π₁	Interpretation	Response
Normal	Normal	Healthy pre-diffusion.	—
Degraded	Elevated	Slow gossip → mempool lagging. The characteristic transient-stress signature (peer churn, bufferbloat, brief network hiccups).	Prefer healthier peers via peer-selection churn; wait it out. Escalate if severity persists or grows (see below).
Normal	Elevated	Pipeline fine yet mempool out of sync. Points at adversarial withholding, a targeted partition affecting only tx-submission for a subset of peers, or a Cardano-specific gossip pathology.	Alert operators; refuse to vote in the affected round; force reconnect to fresh peers. Distinct from network stress and warrants a stronger response.
Degraded	Normal	Slow gossip but overall state healthy — a caching/burst effect, or a lagging monitor whose window hasn't caught up.	No action; sanity-check the transport metric.

The Degraded × Elevated row deserves a severity distinction in practice, scaled by how far the delivery estimate and π₁ sit above baseline. A mild version — delivery time modestly up, π₁ from 0.06 to 0.15 — is the transient signature this row identifies. A severe version — delivery time several-fold up, π₁ from 0.06 to 0.4 — is a compound failure warranting the same response as Normal × Elevated (throttle production, refuse to vote). A raw RTT multiple is at most a cross-check on cause here — a high app-RTT against a healthy kernel RTT points at echo-holding rather than a congested path — not the severity measure itself. Baseline calibration on the testnet is what tells you where "mild" ends and "severe" begins for each region-pair.

The value of the split is that "pre-diffusion is failing" is currently a single conflated signal in the report. A production node armed with the transport-vs-outcome split can react to which kind of failure is in play, and reserve the strongest responses (refusing to vote, alerting operators) for cases that actually warrant them. And this table is the discrete reading of the mesh-stress joint belief (Detecting mesh-wide stress above) over {transport-stress, mempool-fragmentation} — its four quadrants are that belief's modes. A live node runs the belief, with the hysteresis and small-N shrinkage that come with it; the table stays the offline diagnostic snapshot.

Baseline is a prerequisite

The table's "Normal" and "Elevated" categorisations are only useful with a per-region-pair baseline distribution — mean and tail of the tx-delivery estimate (or measured RequestTxs → ReplyTxs completion), mean and tail of observed π₁, all measured under representative load. A week of the testnet campaign's baseline collection (see What a good testnet campaign looks like above) establishes those baselines. Without them, the categorisation is guesswork.

Forced-blended experiments in the next section intentionally drive the mesh into the Normal × Elevated quadrant on the suppressed nodes — their tx-submission is fine with peers that also don't suppress, but their local π₁ shoots up. Confirmatory: the diagnostic table should reproduce the induced failure mode. If it doesn't, either the induction is not doing what you think or the diagnostic is not measuring what you think — both worth debugging.

Investigating fully-blended diffusion when 1-hop fails

Collected and analyzed DeltaQ data fits well as inputs to the report's blended-diffusion model, indirectly but usefully covers the composed end-to-end multi-hop dynamics, and — most valuably — can reveal a family of dynamics the report itself doesn't model.

What "fully blended diffusion" actually requires empirically

The report constructs its blended-diffusion CDF (report.md:359-436) by convolving:

A per-hop transfer-time CDF, parameterised by the throughput model (Mathis or CUBIC) at assumed p and RTT bin (short/medium/long).
A path-length distribution (1..5 hops with the report's regular-random-graph probabilities).
An RTT-bin mixture (short/medium/long equally likely per hop).

Every piece of that has to hold up empirically for the report's conclusion (14 s delivery only 14% under Mathis, 99% under CUBIC) to reflect reality.

What DeltaQ traces give you directly

The single-hop CDF — item 1 above — is exactly what per-connection estBurstS and PeerRTT produce. On a real testnet:

Every burst response (blockfetch chunk, EB body, missing-closure fetch) generates an estBurstS sample: throughput of that hop under the actual congestion regime, actual p, actual buffering. No Mathis/CUBIC assumption in sight.
Every matched cookie produces an RTT sample; a PeerRTT quantile per peer gives you empirical per-hop CDFs binned by peer-distance category.

Plugging those CDFs into the report's convolution instead of the analytical Mathis/CUBIC ones gives you a directly-empirical estimate of the blended-diffusion CDF at each S_EB_tx. That's the cleanest validation path: keep the report's convolution engine, replace its analytical inputs with measurements, compare outputs.

This alone would settle whether the 14% (Mathis) or 99% (CUBIC) figure at 12 MB is closer to reality on a given mesh.

Stratify, and sample the tail. That per-hop CDF is only as good as its conditioning. Its tail — the slow completions that set $F_{\text{full}\mid C}$ — is regime-dependent, so pooling across regimes makes the measured tail a mixture artifact (heavy because a slow regime leaked in, or light because fast ones dominate the count). Bin the per-burst samples by message size, RTT, loss, and congestion control, and build the tail within each bin. This is the per-hop transfer-time variance the leios-conditional-diffusion.md model consumes; the aggregated sample's per-SDU estDeltaQVVar is a different, low-level quantity — within-burst micro-jitter, a CC-smoothness diagnostic (BBR pacing vs CUBIC ACK-clocking), not this input.

The catch is sample count, and it is worse than it looks. The per-hop tail must be resolved deeper than the per-node deficit you ultimately care about: since one slow hop makes a slow path, a per-node deficit of ~1e-4 needs the per-hop tail down to ~1e-4/E[hops] ≈ a few×1e-5 — i.e. ~1e⁵–1e⁶ bursts per bin, from the raw per-burst stream (TraceRecvBurstSDU, shipped selectively per the trace-collection section), not the aggregated 10 s sample. Aggregate volume doesn't rescue it: ~100 nodes × ~30k EBs/week ≈ 3e6 per-hop transfers sliced into dozens of bins is only ~10⁴/bin — short by one-to-two orders in the sparse (rare-RTT/loss) bins that often matter most, with no closed form to interpolate BBR's tail from a few moments (cf. the tail-power caveat).

So reserve the per-hop route for what it is uniquely good at: prediction — convolving the measured per-hop CDF over a different topology (i.e. a different path-length distribution) or a hypothetical CC deployment (the BBR-on-far-peers what-if). For the headline $F_{\text{full}\mid C}$ itself, skip the decomposition and measure the end-to-end per-node arrival CDF directly from the per-EB arrival log: $F_{\text{full}\mid C}$ is a per-EB event (did all $N$ nodes receive this EB by $t$), so ~30k EBs/week resolves its deficit to ~1e-4 by rule-of-three — one composed number, the per-hop variance already baked in over the real paths, no i.i.d. assumption and no per-hop tail to sample.

What only needs a topology snapshot (cheaper than correlation)

One blended-diffusion input is answerable from the peer graph alone — no DeltaQ traces, no per-EB tagging, no consumer plumbing. The main one is structural path-length distribution, the assumption the report leans on hardest for its blended-diffusion CDF (report.md:359-436).

Operational shape:

Query each testnet node's peer-selection state (already introspectable via existing node interfaces).
Dump the graph — edges are directed peer connections; vertices are nodes.
Compute the shortest-graph-path distribution, degree distribution, clustering coefficient, and hub concentration.

What it answers. Does the mesh's structure resemble the report's regular-random-graph assumption (degree ~10, no significant clustering, path-length ~3.6)?

What it does not answer. What paths EBs actually take. Peer-selection preferences and BlockFetch decisions can route diffusion onto longer or shorter paths than the shortest-graph-paths. That gap is filled by reconstruction (next subsection).

Ordering the two. Topology snapshot first, reconstruction second:

Topology snapshot is a one-day exercise. Immediate feedback on whether the mesh's structure resembles what the report assumes. If the answer is "no", that alone is a finding worth reporting before running the more expensive reconstruction. Repeat weekly on a live testnet to catch topology drift.
Reconstruction (next subsection) is the higher-fidelity but more expensive measurement, and its number is what the report's convolution actually depends on.

The comparison between the two produces one of four outcomes, each telling a different story about what should happen next:

Topology says	Reconstruction says	Interpretation	Follow-up
Matches assumed ~3.6	Matches topology	Mesh is what the report assumed; peer-selection isn't biasing diffusion.	Report's convolution well-founded; confidence in 12 MB feasibility increases.
Matches assumed ~3.6	Longer than topology (e.g. ~4.8)	Structure is fine but peer-selection pushes diffusion onto longer paths.	Investigate peer-selection behaviour; unexplained bias in the diffusion routing.
Longer than assumed (e.g. ~5.2)	Matches topology	Mesh structure is worse than assumed; diffusion follows what the structure allows.	Recalculate the report's convolution with the empirical path-length.
Longer than assumed	Shorter than topology	Peer-selection is routing around long paths (via hubs, for instance).	Model has a hidden resilience mechanism worth documenting.

Doing only one of the two measurements leaves ambiguity that the other resolves.

What requires cross-node correlation

The convolution assumes hops are independent and identically distributed. The mesh may not honour that. To check, you need to observe end-to-end delivery, not just single-hop:

Tag every EB (or synthetic bulk payload) with a stable ID, log (node, ts, EB_id, arrival_or_forward) events across the whole testnet.
Also tag each BlockFetch / LeiosFetch fetch session at each node with a local session ID that references the EB hash it's pulling. The EB-arrival log above is application-layer and answers "when did EB X arrive at node Y"; a per-fetch session tag on the local traces answers "which mux-layer signal belongs to which fetch". Under the interface-saturation scenario in Dynamics the report doesn't model (item 1), this makes the cross-node correlation direct: "during session $S$ for EB $X$ on connection B↔C, B's other Wantons saw oldest-byte-age rise by $\Delta$, and peers D/E/F saw estBurstS drop by $\varepsilon$" — instead of timestamp-guessing which spike belongs to which fetch. No wire-format change needed: the EB hash is already on the wire (BlockFetch / LeiosFetch requests reference EBs by hash), so the session ID can be derived deterministically from (peer, EB_hash, fetch_start_time) and only needs to appear in the local BlockFetch / LeiosFetch client trace stream to key mux-layer signals against.
Reconstruct actual paths by matching each node's arrival time against its forwarding events, plus the known peer graph. You now have the actual multi-hop trajectory of each EB across the mesh.
Compare the observed end-to-end arrival CDF against the DeltaQ- convolved prediction. If they diverge, the independence assumption is wrong — head-of-line effects at intermediate hops, or graph-topology skew, are the likely culprits.

This isn't in the mux traces themselves — it's a higher-level analysis layered on top. But the mux traces plus a modest per-EB-arrival log gives you enough to do it. cardano-tracer already emits block-arrival events; the same pattern for EBs would suffice.

Measuring $F_{\text{full}\mid C}$ directly (Yves Hauser's §5.6 target)

The same per-EB arrival log that enables path-length reconstruction also lets us directly measure Yves Hauser's §5.6 conditional probability: given certification succeeded at $t_v = 7\text{ s}$, did all $N$ honest nodes receive the EB body by $t = 14\text{ s}$? See leios-conditional-diffusion.md for the full analysis; the operational recipe is:

Log (node, EB_id, arrival_ts) at every measurement node (same log as the cross-node correlation section above).
Log certification outcome per EB (from cardano-tracer's block-diffusion / EB-certification traces).
For each EB $j$ with $C_j$ (certification succeeded), check whether $\max_i T_j^{(i)} \le 14\text{ s}$.
Empirical estimator: $\hat{F}_{\text{full}\mid C}(14) = \frac{\left|{j : C_j \text{ AND all arrived by 14 s}}\right|}{\left|{j : C_j}\right|}$.

Yves's formula predicts $\approx 1.000$ at the report's default parameters — for the body-only, honest-case question. The testnet gives us an empirical rate. (Closure coverage and the adversarial $G_{\text{adv}}$ case are separate measurements; see leios-conditional-diffusion.md.) Detection tail power: ~30 k EBs/week makes the coarse gradations (0.9 / 0.99 / 0.999) all trivially distinguishable; zero observed failures bounds the true failure rate at $\lesssim 3/n \approx 10^{-4}$ (rule of three), resolving $F$ to ~0.9999 — finer needs ~10× more EBs per nine, and failure clustering (gap 4) shrinks the effective sample (see the tail-power caveat in leios-conditional-diffusion.md).

As a by-product, the same data gives:

The identity of the slowest node per EB ($\arg\max_i T_j^{(i)}$) and the temporal regime of that identity across rounds (persistent / rotating / bimodal). Per-node operational intervention pays off only under the persistent regime; the rotating and bimodal (churn-induced) regimes each call for different levers. See leios-conditional-diffusion.md's "Slowest-node identity and its temporal structure".
Empirical vs modelled $G(t)$ — divergence tells us whether $G$ is over- or under-estimated at each body size.
The correlation matrix between per-node arrival times — directly addresses Yves's caveat 1 (i.i.d. assumption) and gap 2 of leios-report-gaps.md.

Measuring churn-induced peer correlation and orphan-set geography

The appendix in leios-fetch-scheme.md identifies a tendency for peer-selection churn to concentrate the mesh's big-ledger peer sets around block-production regions, producing bimodal per-round coverage (high on dense-origin rounds, low on sparse-origin rounds). The actual strength of this tendency on real networks is an empirical question. The same per-EB arrival log used for $F_{\text{full}\mid C}$ measurement (above), extended with peer-metadata and block-origin metadata, supports the validation.

1. Per-round orphan-set identification. For each certified EB, identify the set of nodes that did not receive delivery from a big-ledger seed (their winning arrival was via a non-big-ledger path, or arrived after a threshold indicating multi-hop). Peer-metadata (big-ledger flag per connection) distinguishes the winning path.

2. Block-origin classification per round. For each certified EB, identify the block producer's geographic / ASN region. Classify the round as dense-origin or sparse-origin based on the producer's location (or, more robustly, based on stake-weighted region density in a rolling window). This is the key axis of the analysis.

3. Per-origin coverage rates ($c_D$, $c_S$, empirical $k$). Compute mesh-wide seed-hop rate separately for dense-origin and sparse-origin rounds:

$c_D$ = mean mesh-wide seed-hop coverage (fraction of nodes reached by a big-ledger seed) on dense-origin rounds.
$c_S$ = same for sparse-origin rounds.
Empirical $k = c_S / c_D$.

(This is the same estimand as in leios-fetch-scheme.md's appendix — a per-round-type mesh-wide coverage fraction, not a fraction of rounds.)

If $c_D \approx 85%$ and $c_S \approx 40%$, then $k \approx 0.47$ — strong-ish concentration. If $c_S \approx 75%$, then $k \approx 0.88$ — weak concentration, the tendency isn't dominant.

4. Per-round bimodality analysis. Instead of asking "which nodes are persistently orphaned", ask "which rounds have widespread orphaning". Histogram the mesh-wide orphan fraction across all rounds. Under the churn tendency, expect a bimodal distribution (peak near zero for dense-origin rounds, peak near 0.75–0.8 for sparse-origin rounds). Under weak tendency, expect a unimodal distribution centered on the mesh average.

5. Cross-node covariance within rounds. On sparse-origin rounds, compute the covariance between arrival times across nodes. High within-round covariance is the signature of a shared bottleneck (the multi-hop diffusion route from a sparse origin). This directly measures Yves's caveat 1 applied to the seed-node lottery.

6. Peer-graph diversity metrics (secondary). For each node, compute ASN / country / geographic diversity of the 5 big-ledger peers, peer tenure, and average RTT. Correlate diversity with per-node orphan-frequency. Under the tendency, diversity should correlate positively with fewer sparse-origin orphanings; under weak tendency, no strong correlation.

7. Effect on $F_{\text{full}\mid C}(14)$ split by round type. Compute empirical $F_{\text{full}\mid C}(14)$ separately for dense-origin and sparse-origin rounds. If sparse-origin $F_{\text{full}\mid C}$ is significantly below dense-origin, the tendency is materially degrading protocol safety on those rounds. Operational trigger for deploying Improvement #8 (diversity-aware churn) or accepting the reduced robustness on sparse-origin rounds.

Measurement pre-requisites

Peer-metadata service — MaxMind or equivalent for ASN / country lookups from IP addresses. Roughly a few MB static database, daily refresh. Cheap.
Peer-graph tracking — per-node record of which peers were big-ledger vs NBL at each round. Small addition to the per-EB arrival log.
Block-producer identity per round — required to classify rounds as dense-origin or sparse-origin. Available from the block header / chain-sync metadata already emitted by cardano-tracer; needs to be joined into the per-round dataset.
Regional partitioning of stake — a rough "dense vs sparse" region cutoff for classifying block-origins. Could be geographic (continents), by-ASN, or by empirical stake density. Multiple partitionings for robustness against a bad partition choice.

Comparison across configurations

Under the four-configuration experiment (item 6 of "What a good testnet campaign looks like"):

Memo baseline: establish empirical $c_D$, $c_S$, $k$, and the per-round bimodality profile. If $k \approx 1$ (no strong concentration), the tendency isn't dominant on this mesh and the other configurations may not add much. If $k \ll 1$ (strong concentration), we have a real problem to fix.
+Improvement #1 (hybrid classification): minimal effect on churn dynamics — classification refines within the existing peer set but doesn't change churn selection. Concentration should be similar to baseline.
+Improvements #1–#8 (full adaptive): Improvement #8's diversity-aware churn should measurably raise $c_S$ (sparse-origin coverage) and hence $k$. Regional variance in $F_{\text{full}\mid C}$ across round-types should shrink.
Erasure-coded variant: orphaning behaviour differs entirely (no single seed peer needed). Different question — not directly comparable but useful as an alternative.

The dominant question the measurement answers: how much does diversity-aware churn actually raise coverage on sparse-origin rounds? If the answer is "meaningful gain" ($k$ climbs from 0.5 to 0.8), the mitigation is worth deploying; if "marginal" ($k$ climbs only 0.5 → 0.55), the complexity may not be justified.

What requires inducing the scenario

Blended diffusion only matters when 1-hop pre-diffusion has failed. To exercise it on a testnet you have to force pre-diffusion to fail:

Suppress tx-submission on a controlled subset of nodes (or the whole mesh) for a window. Every EB in that window has to be blended-diffused.
Introduce artificial mempool divergence by having a fraction of nodes accept transactions from a private endpoint that others don't see. π₁ rises artificially; blended-diffusion becomes the norm.
Withhold: an adversarial-simulation variant where a bloc of nodes deliberately don't forward tx-submission but do forward blocks. Tests the report's §7 limitation 5 (adversarial withholding).

Under any of these, DeltaQ traces then show you the actual blended-diffusion dynamics under load. The report predicts what should happen; the trace shows what does.

Dynamics the report doesn't model, but DeltaQ traces expose

This is where DeltaQ trace collection is most valuable, because it can challenge the report's framing rather than just confirming its numbers.

Inter-connection interface contention at forwarding nodes. When node B is forwarding a 12 MB closure from A to C, the fat LeiosFetch flow to C consumes a large fraction of B's outbound interface bandwidth. B's concurrent thin flows on other connections — tx-submission to D, chain-sync to E, KeepAlive echoes to F — share the interface with the fat flow. This is a distinct mechanism from the intra-connection head-of-line blocking covered in Vote diffusion, head-of-line blocking, and Praos priority below: SDU-level round-robin keeps scheduling into the bearer fair, but bytes still couple below the mux (batching, kernel send buffer, TCP loss-recovery, reply ordering) on any single shared connection. The concern here is different again — inter-connection competition for B's shared network interface across its many connections, which degrades the thin flows only if the interface is genuinely saturated. Detecting this can go via cross-node correlation or local self-monitoring at B, using different signals:
- Cross-node. estBurstS is receiver-side — each of B's peers logs its own estBurstS for traffic from B, and a simultaneous drop across multiple peers of B during B's bulk-transfer window is the signature.
- Local at B (self-monitoring). Under NIC saturation, kernel TCP throttles B's socket-to-D (the thin flow), the socket's send buffer fills, and the mux write from the tx-submission Wanton blocks — so bytes accumulate in the Wanton and oldest-byte-age on B's per-mini-protocol egress queue grows. A rising oldest-byte-age on B's thin-flow Wantons during B's fat-flow window is the local signature of interface saturation — per-mini-protocol, per-connection granularity, exactly what we need. The dual signal is socket write-block latency: time the mux egress thread spent blocked on a socket write with the kernel send buffer full — the mux-visible echo of the kernel-side congestion. Both traces are proposed in track.md's "Measurements blocked on missing traces" (items 2 and 6) but not yet emitted; adding them would give B direct self-monitoring for this scenario, covering both sides of the stall (queue backing up, mux blocking on the socket). Complementary signals available today: aggregate outbound bandwidth from OS-level interface counters, and TCPInfo (tcpi_snd_cwnd, tcpi_retrans, tcpi_lost) which shows the kernel-side congestion state driving the mux stall.
The report models per-hop transfer time as if each connection has independent access to the wire (see leios-report-gaps.md gap 5 — per-node interface contention across concurrent connections is not modelled).
Round-to-round feedback. A round with heavy blended diffusion consumes mesh bandwidth that would otherwise carry tx-submission, which raises π₁ for the next round, which raises the blended-diffusion load again, which... The report treats rounds as independent Bernoulli trials. Time-series DeltaQ traces across many consecutive rounds show whether the mesh has this positive-feedback autocorrelation. If it does, the tail probability of consecutive-round failure is worse than the report's Bernoulli model implies.
α as a time-varying observable rather than a scalar. The report defers α (probability that pre-diffusion is operating normally) to a single number in the α · P_cert,1-hop mixture, and treats it as a system-wide constant. But at each moment, per node, the observed fraction of arriving EBs whose π₁ stays below the 1-hop threshold is a direct sample of the α distribution — computable from EB arrival processing joined against local mempool state. That converts a hidden static parameter into a time series with a distribution, a tail, and cross-node covariance — the last of which is the important one: whether pre-diffusion failures hit the mesh in lockstep or independently is the difference between a global P_cert collapse and quorum still meeting from healthier nodes, and the report's single-scalar α silently assumes the lockstep case.
Recovery dynamics. After a pre-diffusion failure, how does the mesh recover? Do voters flag the miss and re-fetch aggressively? Does the tx-submission backlog burst-catch-up? DeltaQ traces around the failure event show the recovery signature — most importantly, whether recovery is monotone or oscillatory. Monotone recovery is safe; oscillation is a warning about latent instability.
Storm dynamics under concurrent EBs. Nothing forces exactly one EB per round to be certified concurrently — there could be multiple candidates propagating simultaneously. DeltaQ traces during multi-EB windows probe whether the mesh gracefully shares bandwidth or whether one EB starves the others.
Path-length skew. The report assumes a regular random graph. Real testnets have topology bias (regional clustering, hub nodes, degree variance). Reconstructed EB paths from cross-node correlation reveal the actual path-length distribution and whether it matches the report's assumed distribution. If typical paths are actually 4.5 hops rather than 3.6, the whole convolution shifts.

Cascade to livelock: bounding sustained-overload failure

Item 2 above (round-to-round feedback) names the mechanism; this subsection names the pathological outcome and enumerates the measurements that bound it.

The failure mode. Sustained mempool arrival at or above the level at which max-size EBs are produced imposes network cost (EB body diffusion, closure fetch, vote diffusion, certRB diffusion) and CPU cost (apply/reapply for validation) each round. If certification fails at that operating point, the work is not wasted — TxCache retains the TX bodies for reuse — but the transactions themselves stay in the mempool, the next producer sees a similar-or-larger backlog, and produces another max-size EB. If the reason certification failed is a saturation in the components TxCache does not amortize (network capacity for EB body / votes / certRB, per-node validation CPU, slow-node tail dominance of $F_{\text{full}\mid C}(14)^N$), the next round's certification fails for the same reason. The mesh is doing work each round but not making protocol progress — a livelock.

The mechanism behind this — why the work-side self-damps (mempool backpressure plus the retained closure cache), where a genuine sustained failure lives instead (route (a), the non-tx / voter-saturation channel, versus the self-damping work-side route (b)), the adaptive-sizing lever, the Praos fall-through floor and its dependence on Praos-over-Leios mux priority, and the adversarial two-tier form — is developed in leios-queueing-theory.md's appendix. This section keeps the problem statement above and the measurements that bound it below.

Measurement recipes

Ordered from cheapest (existing traces) to most involved (induced- failure experiments).

Detecting the feedback loop. Available today or with minimal instrumentation.

Per-round certification outcome autocorrelation. From cardano-tracer's EB-certification traces, compute $\mathrm{Corr}(\mathrm{fail}(R), \mathrm{fail}(R+k))$ for $k = 1, 2, \ldots$. Positive autocorrelation is the cascade signature; the decay rate quantifies memory depth. The independent-Bernoulli null hypothesis gives $\mathrm{Corr} = 0$.
EB body size vs prior-round outcome. Cross-tabulate round $R+1$'s body size against $\mathrm{fail}(R)$. If body size systematically grows after a failure, that's the direct feedback signature. Available today from block-production traces.
Mempool occupancy trajectory across consecutive-failure runs. Track depth and capacity: reaching capacity and staying there is the full-mempool backpressure regime (expected under sustained overload — new admissions defer and inbound tx-pulls quiesce), not by itself the runaway; the runaway question is answered by the self-damping series below, not by depth. From the mempool-measurements dataset, which already carries a static hint of the load→π₁ coupling: its us-east-2 node shows π₁ ≈ 0.44 at >85% utilisation versus ≈ 0.06 at low load — cache-miss rate (and per-round closure-fetch cost) climbs sharply with utilisation, the direction that drives the work-side load up before caching damps it.
Self-damping vs sustained — the per-round series. The work-side signature is whether the reactive load decays. Log, per round: the closure cache-miss fraction, closure-fetch bytes split hit/miss, TxCache occupancy, and the inbound-tx-pull backpressure flag. A miss fraction falling round over round while pulls are backpressured off is the benign self-damping stall; one that stays flat or climbs is a non-self-damping channel — disjoint-tx cache-locality erosion, or a route-(a) failure caching cannot touch — which is the case that actually sustains. It is also what distinguishes a decaying failure-autocorrelation (self-damping) from a persistent one (route (a)).

Quantifying TxCache's role. Splits the retry-cost savings attributable to TxCache from other sources.

Per-node TxCache hit rate, split by "TX belongs to a failed EB from round $R-k$" vs "TX arrived via ordinary tx-submission". Reveals what fraction of retry savings is TxCache's contribution.
Closure size on retry. For consecutive EBs sharing overlapping TX sets, measure the actual closure-fetch bytes pulled at each node. Reduction vs full closure is TxCache's realised savings — per node, per round, per overlap fraction.
CPU cost of apply/reapply on retry, split by cache-hit status. If TxCache caches validation state, retry cost should drop sharply; if only TX bodies, revalidation cost is largely unchanged. Uses existing post-cip/apply-reapply instrumentation.

Finding the collapse threshold and recovery signature. Controlled experiments that induce the failure mode.

Sustained-load-to-collapse sweep. Increase tx-submission arrival rate until certification failure rate crosses a threshold (e.g., >5% consecutive failures over a rolling window). Drop the arrival rate back to baseline and measure whether certification recovers. If certification stays stuck after load drops → livelock confirmed. Report the smallest load level above which recovery does not occur (the "livelock onset" boundary).
Induced-failure recovery experiment. Force a round to fail — withhold votes at a controlled slice of the committee, or inject a controlled network shock via tc netem — then observe recovery: how many rounds until certification returns to baseline, and whether the trajectory is monotone or oscillatory. Signatures per item 4 above. This also separates the two adversarial tiers: hold a producer subset at max body size and confirm honest rounds still recover (cheap, self-healing tier); add continuous saturation of a committee slice (tc netem) plus withheld votes and confirm recovery does not occur (expensive tier) — the livelock-onset boundary should appear only with the continuous channel present.
Time-to-recovery distribution. Across many induced failures at different load levels, measure $\Pr(\text{recovery within } k \text{ rounds})$ as a function of load. A regime where recovery time diverges beyond the tx-arrival timescale is where livelock lives.
EB-issuance-rate (production-gap) sweep. Vary the EB rate — or inject synthetic long gaps — while holding a producer subset at max body size (the work-side kick), and measure recovery probability against rate. Per threat-model-additions.md's T42, a sparse (Praos-cadence) rate should let the work-side collapse heal across the next long interblock gap with no lever, whereas a dense rate makes healing gaps rare and the collapse persists — turning the EB-density dependency from a stated open question into a measured onset curve. Run it with the continuous voter-saturation channel off (the gap heals) and on (it does not) to separate the work-side route from the schedule-independent one.

Tuning the stabilizing lever. Once producers use adaptive body sizing, calibrate its response.

Adaptive body sizing calibration. At the collapse threshold, how much body-size reduction restores certification? Report the operating curve $S_{EB\text{-tx}}(\mathrm{load})$ that keeps the certification success rate above a target.
Adaptive-sizing loop stability. When producers throttle, next-round body size shrinks, certification recovers, mempool grows, next-round producer sees lower degradation, body size grows again. Whether this loop converges to a fixed point or oscillates is the open question flagged in leios-conditional-diffusion.md's follow-ups. Measure the loop-gain from perturbation experiments.
Voter self-throttle activation rate. How often does the self-throttle fire, and does the fire rate correlate with load? A throttle firing so often that certification never happens is a different failure mode (throttle-induced stall) but it's how the mesh avoids the livelock. Measurable from voter-participation traces once the runtime lever is enabled.

Cross-referenced signals from other measurements.

Per-EB arrival log with certification outcome — the same log powering $\hat{F}_{\text{full}\mid C}(14)$ (per leios-priority-measurement.md).
estBurstS and PeerRTT degradation cross-referenced against the failure-autocorrelation series — links the transport-layer signal to the round-outcome signal.
Interface-saturation self-monitoring (oldest-byte-age and socket-write-block latency, per track.md items 2 and 6) during heavy-round windows — signals whether the network layer is the binding constraint.

Traces for cascade replay. The analyses above are point measurements, mostly from existing datasets; replaying an episode — re-running it under a counterfactual TxCache size, EB-issuance rate, lever setting, or mux priority — needs the cascade's inputs logged per node per round: the same "log the inputs, not the output" principle as Traces for offline study, but for the cascade rather than the belief. One record per node per round (heavy — sample it, or enable it only during induced-failure runs), on the shared monotonic clock with a slot/round ref:

mempool — occupancy vs capacity, and a tx-id-set digest (rolling hash or bloom) so cross-node fragmentation is reconstructable;
tx-submission — the inbound-pull backpressure flag and outbound serve activity (the mechanism: pulling quiesces on a full mempool, serving does not);
the EB seen or produced — its referenced tx-id set (or digest) and advertised size (feeds the disjoint-tx / overlap analysis);
closure-fetch — bytes and tx count split cache-hit/miss, T_wait, and arrived_from_peer;
TxCache — occupancy, hit/miss counts, and evictions (the bounded-cache gap);
the deadline budget — time consumed by EB-body diffusion, closure-fetch, apply/reapply, vote collection, and certRB diffusion, each against the round deadline, so a failure attributes to the work-side or to a route-(a) component;
the base chain (RB) — whether the round's ranking block carried a certificate or fell through to carrying txs directly, and in the fall-through case the tx count / bytes it drained (the Praos floor engaging, and at what rate — distinguishes healed at the floor from above the floor);
base-chain diffusion vs the Praos bound — the RB's own diffusion time (cert- or tx-carrying) against Praos's ~5 s Δ, so a crash through the floor — the base chain itself missing its bound — is visible, not just a Leios-layer failure;
egress contention — the shared bearer's state (queue depth / oldest-byte-age / socket-write-block latency, per the cross-referenced track.md items, but recorded per round) and the RB-vs-closure sharing on it, so a delayed RB attributes to self-inflicted bufferbloat (Leios closures crowding the base-chain block) rather than external degradation;
outcome — certified / failed + mode; for a voter, voted / missed-deadline / withheld.

With this, an episode replays as a simulation seeded by the logged arrivals, EB contents, and topology: it confirms the mechanism (self-damping versus a sustained channel), attributes the binding constraint, separates a collapse that healed at the Praos floor from one that crashed through it, and answers the counterfactuals — would a larger cache, a sparser EB schedule, the lever, or Praos-over-Leios mux priority (WFQ / a separate bearer) have changed the outcome — that the point analyses alone cannot.

Cross-references

leios-report-gaps.md gap 4 — the modelling gap this analysis is a downstream consumer of.
Item 2 (round-to-round feedback) and item 4 (recovery dynamics) above — the surrounding dynamics this section makes concrete.
leios-conditional-diffusion.md lever 2 — the actual stabilizing lever this section relies on.
leios-overlap.md's "Where the fit is not clean" — treats TxCache as a scalar π₁, which is the framing this section extends into dynamic-stability terms.
leios-queueing-theory.md — reads this whole failure mode through queueing/performance theory (non-work-conservation, metastability, the reflecting Praos floor, regeneration via production gaps, order-statistic fragility), giving the narrative here its structural underpinning.

The practical shape of a "blended-diffusion investigation" campaign

If this is your goal, the testnet setup differs slightly from the plain validation campaign above:

Explicit forcing knob: a scripted way to enable/disable tx-submission per node. Cardano-node already supports peer-filtering hooks that could do this.
Per-EB arrival log: augment cardano-tracer output to include (node, EB_id, arrival_ts, forward_target_ts) records. Not currently emitted per-EB but the trace event exists at block-diffusion level.
Round-cadenced captures: raw DeltaQ traces on measurement nodes during a specific window around a forced-blending event, not just steady-state sampling. Higher temporal resolution during the interesting events, aggregated the rest of the time.
Multi-round runs: at least 100 rounds under each of {full pre-diffusion, forced-blended, adversarial-withholding} to get tail estimates. The report's rare-event probabilities (14%, 31%) need substantial sample sizes to validate.

Two limits you can't get past

Tail estimation. The report's most stressed predictions are rare-event certification failures. A week-long testnet with ~100 nodes at 1 EB/20 s gives ~30 k certified EBs; with zero failures observed that bounds the failure rate at $\lesssim 10^{-4}$ (rule of three), so the campaign resolves $F_{\text{full}\mid C}$ to ~0.9999. Rarer events (a $10^{-5}$ failure rate is ~0.3 expected failures/week) stay paper-only unless you run longer or with more nodes — and failure clustering (gap 4) shrinks the effective sample further.
Testnet vs mainnet mesh topology. Even if you match the number of nodes and regions, real SPO topology has network-of-network effects (relays behind stake pools, edge nodes on residential uplinks, community relays), that a controlled testnet doesn't reproduce. The blended-diffusion path distribution on mainnet may look quite different from a synthetic mesh. This is the same caveat as before but bites harder for blended-diffusion because path-length skew propagates through the convolution multiplicatively.

The short answer

DeltaQ traces give you (a) the single-hop CDF inputs to the report's convolution directly, (b) with modest additional per-EB arrival logging, the end-to-end multi-hop composed behavior, and (c) — most interestingly — exposure to dynamics the report doesn't model at all: inter-protocol contention, round-to-round feedback, recovery signatures. To exercise blended diffusion specifically, you also need a way to force pre-diffusion to fail on the testnet. The report's static-scalar α becomes a measurable time series once you're collecting traces this way, which is a meaningful qualitative upgrade to the analysis.

Vote diffusion, head-of-line blocking, and Praos priority

The validation campaign as described logs EB body arrivals and certification outcomes but not votes, and the report models no vote traffic at all (votes enter only through the sortition math). That leaves a hole: when certification fails or lags on the testnet, we cannot attribute it — was the EB body late to voters, were the votes late back to the certifying producer, or were votes stuck behind bulk on a shared bearer? This section adds the vote-path measurement and the intra-connection head-of-line story it depends on. It is the measurement counterpart to leios-report-gaps.md gap 17.

Why votes need their own instrumentation

The 7 s voter deadline (3·L_hdr + L_vote) and the 450-of-600 quorum are a round-trip budget: the EB body must reach voters, they must validate and vote, and 450 votes must reach the certifying producer — all inside the window. Yves's $F_{\text{full}\mid C}$ measures only the first leg (body arrival at all nodes). Votes are their own network traffic — diffused via LeiosNotify offers + LeiosFetch MsgLeiosVotesRequest / MsgLeiosVoteDelivery, small and latency-critical — and they share bearers with the 12 MB closure flows. Add per-vote arrival logging: (election_id, voter, node, ts) from the LeiosNotify/LeiosFetch trace stream, so the vote-diffusion CDF becomes a first-class measured object alongside the body CDF. Cheap (votes are small, the trace points exist), and it closes the other half of the certification-timing budget so a failed round is attributable rather than merely counted.

The intra-connection head-of-line couplings

The mux's SDU-level round-robin is often summarised as "mini-protocols can't block each other on a connection". That is true only of scheduling into the bearer. Below and after that point, four couplings still head-of-line block latency-critical traffic (votes, Praos headers) behind a 12 MB closure on a shared connection:

Egress batching. buildBatch submits up to batchSize (128 kB, or 100 SDUs) per writeMany; an SDU arriving mid-batch waits behind the whole write. Within a batch protocols interleave fairly — the SDU that didn't fit is the one that waits — and the batch size can be tuned down if this dwell is material.
Socket send-buffer FIFO. The egress thread issues a blocking sendAll/sendMany; with kernel wmem autotuned to MB scale on high-BDP paths, a small SDU written "fairly" still sits behind megabytes of already-queued closure bytes, draining at path rate (hundreds of ms to seconds). TCP_NOTSENT_LOWAT (Linux/Darwin, precedented — Nick's mininet test-bed sets it at 512 kB) caps the not-yet-sent backlog the kernel accepts before blocking the writer, cutting the dwell from "wmem ÷ path rate" toward "≈ one BDP ÷ path rate". Costs: the egress thread wakes more often, and the cap must stay ≳ BDP to keep the pipe full. Worth exposing as a bearer option (default off, recommended for Leios bulk connections).
TCP loss-recovery. In-order delivery stalls every mini-protocol on the connection for ~RTT (fast retransmit) up to an RTO on a single loss. A 12 MB closure is ~8,600 packets; at p = 1e-4 that is ~0.9 expected loss events per transfer (~9 at the 1e-3 evening-peak rate), so roughly one such stall per closure per connection at baseline loss. Crucial nuance: two LeiosFetch instances on the same connection (below) escape reply-ordering HOL but not this coupling — only a separate connection does. NOTSENT_LOWAT doesn't help here either; it's a delivery-side, not a queue-ahead, stall.
Intra-protocol reply ordering (the one the CIP flags). typed-protocols' granular states force a LeiosFetch server to reply in request order, so a vote reply queued behind a 12 MB closure reply in the same LeiosFetch instance is head-of-line blocked — "threatening freshest-first delivery or even motivating inflations of L_vote and/or L_diff" (CIP-0164:1436-1439).

Congestion control is orthogonal to most of this. A recurring hope is that switching the bulk flow's controller (CUBIC → BBR) relieves the couplings. On a shared bearer it mostly does not: CC governs the rate the socket drains, not the order of bytes within one sequence space, so it cannot let a vote jump ahead of closure bytes the mux already queued in front of it. Couplings 1, 2 and 4 are byte-ordering effects, untouched by the controller. CC touches only coupling 3, and only indirectly through loss/retransmit frequency: on a non-congestive-loss path BBR drains the closure bytes ahead of a vote faster and stalls in-order delivery less often — a latency gain, but via the throughput mechanism, not queue discipline — while on a congestive bottleneck the advantage erodes or inverts: BBR's overshoot (STARTUP dominates for these short transfers) is loss-blind (v1; v3 less so), so it can sustain loss rather than backing off as CUBIC would, and the extra loss feeds coupling 3's stalls. The bufferbloat resistance that makes BBR a good latency neighbour is a cross-flow effect: it needs the vote traffic on a separate flow to bite (see connection separation), not a shared bearer.

Two-instance LeiosFetch: measure the mitigation

The CIP's own suggested fix for coupling (4) (CIP-0164:1439-1443): "run two instances of LeiosFetch and reserve one for requests that are small and urgent (e.g. small blocks, a few missing transactions, or perhaps any vote); the existing infrastructure would naturally interleave those with the larger and/or less urgent requests." That last clause is the mux — the SDU round-robin (WFQ on the perf branch, below) interleaves the two instances so the small/urgent instance's SDUs are never stuck behind the bulk instance's replies. This is a protocol-structure problem whose solution is a mux-scheduling arrangement, so it sits squarely in this repo's scope.

The experiment (needs the vote-path log above): deploy both arrangements — single-instance LeiosFetch vs two-instance — and measure vote-arrival latency at the certifying producer under heavy closure load. Prediction: single-instance shows vote replies tailing behind 12 MB closure replies (a bimodal vote-latency distribution, the slow mode aligned with concurrent closure fetches); two-instance flattens it. A clean, self-contained validation of a CIP recommendation using this branch's machinery. Note the split fixes reply-ordering (4) but not loss-coupling (3) — for that the urgent instance must be on a separate connection.

Composing with mux WFQ

Scheduling between the two instances (and between Leios and Praos generally) is what an open draft PR (branch origin/mw/mux-single-peer-performance) adds to the mux: weighted fair queuing over egress queues, plus a per-protocol burst token bucket. WFQ's proportional-share semantics maps directly onto the CIP's tolerant ("does not need to be perfectly strict", CIP-0164:1187-1209) Praos-over-Leios priority — strict priority would risk starving Leios entirely; weights don't. The full priority stack is therefore: WFQ (scheduling between protocols/instances) + two-instance LeiosFetch or experimental typed-protocols server-side reordering (reply ordering) + NOTSENT_LOWAT (kernel buffer, coupling 2) + connection separation for the most latency-critical traffic (loss coupling 3). One open design question for WFQ: the certified-EB urgency inversion (a certified EB becomes as urgent as the RB it blocks, CIP-0164:1193-1198) makes the right weight time-varying, so either weights must be runtime-adjustable per protocol instance, or urgent certified-EB fetches go on the high-weight instance. See leios-report-gaps.md gap 17 for the gap framing; none of these pieces is merged to main yet.

This weighting is part of what the sustained-overload Praos fall-through floor needs: on the default fair round-robin a large closure bufferbloats the egress and can delay the fallback base-chain block meant to floor a livelock at Praos, so Leios overload can degrade the base chain rather than merely degrade to it. WFQ schedules the base-chain SDUs ahead, but cannot undo a closure already ahead in the same connection's TCP sequence space — that residual needs the connection separation below (or QUIC). Until both land, the floor is not guaranteed — see the Praos floor in leios-queueing-theory.md's appendix.

Connection separation for latency-critical traffic

Of the four couplings, only #3 — TCP loss-recovery, where in-order delivery stalls every mini-protocol on the bearer until a lost segment is retransmitted — survives every same-connection mitigation. WFQ, two-instance LeiosFetch, and NOTSENT_LOWAT all act above one TCP sequence space, so a loss still stalls everything sharing it. Escaping #3 means putting the smallest, most latency-critical traffic — votes above all, arguably Praos headers — on a separate channel with its own sequence space. This is the heaviest hammer in the stack; reserve it for traffic where a loss-induced stall of the bulk flow would blow a hard deadline (votes race the 7 s certification budget).

Three ways to separate, in increasing divergence from today's model:

A second TCP bearer per peer — small/urgent protocols on one socket, bulk LeiosFetch on another. Fully escapes #3 (the urgent socket has its own sequence space, so a closure retransmit can't stall a vote) while keeping TCP's reliability, congestion control, and the mux model unchanged. Cost: a second connection's worth of kernel state, handshake, versioning, and teardown per peer, and the two flows compete at the bottleneck as independent TCP flows (each with its own cwnd — fine here, since the urgent flow is tiny and wins its share easily unless the NIC is genuinely saturated).
UDP for the urgent traffic. Votes are an unusually good fit: each is a single sub-MTU datagram (a BLS sig plus a few IDs, well under 1500 B), self-authenticating (signed — forgery is caught), and gossiped by fan-out where some loss is tolerable (you need 450 of 600; a datagram dropped to one peer is recoverable from another, or by re-request). UDP then buys exactly what we want: no head-of-line blocking (each datagram independent — a lost one never stalls the next), no per-message connection state, no handshake, no send-buffer bufferbloat behind bulk. The complications bite hardest here:
- No network-usage bounding. TCP self-limits via cwnd; UDP does not. We would have to add application-level pacing / rate-limiting, and getting it wrong feeds the very congestion the bulk traffic already stresses (or self-DoS). Unbounded UDP is also a spoofing / reflection surface — signed votes catch forged content, but processing spoofed datagrams (verify cost) is itself a DoS vector needing source validation and rate caps.
- No reliability or ordering. "Did I get the votes I need?" becomes application-level accounting, not a TCP guarantee. Votes tolerate this (quorum + redundant senders); Praos headers do not (you need the header to fetch the block), so headers want a reliable separate channel, not raw UDP.
- NAT / firewall traversal. Many SPO deployments permit only the established TCP node-to-node port; a UDP side-channel needs its own firewall rules and hole-punching — a network-wide operational change touching multi-implementation consistency (gap 13) and the operational-vs-spec boundary (gap 16).
- Outside the mux telemetry. DeltaQ / cookie-RTT / estBurstS ride the TCP bearer; a UDP channel is invisible to them, so the very signals this campaign is built on wouldn't cover the urgent path — parallel instrumentation needed.
QUIC — the principled middle ground: independent streams within one UDP-based connection, so a lost packet stalls only its stream (escapes #3 like separate connections) but with built-in congestion control (userspace, per-connection — CUBIC/BBR selectable, not fixed; one controller per connection, not per-stream), reliability, and encryption (unlike raw UDP) over a single connection (unlike a second TCP bearer). One residual: the shared cwnd means QUIC escapes the delivery-order stall (#3) but not a connection-wide rate coupling — a loss cuts the whole connection's window, and bulk saturating it can briefly (~1 RTT) delay an urgent send. But that is soft and priority-schedulable (the scheduler hands the window to the urgent stream; bulk absorbs the cut; leaving cwnd headroom mitigates it) — a mild #2, not the hard #3. Cost: adopting QUIC in a stack whose mux is TCP today — a large change — but it is the option that solves the head-of-line coupling without the raw-UDP bounding problem or the two-connection overhead. But being UDP-based it inherits raw UDP's reachability caveat: a firewall permitting only the TCP node port blocks QUIC too. It handles NAT-binding maintenance better than raw UDP (built-in keep-alives, connection migration), but the binary "is UDP allowed at all" it cannot escape. In Cardano's permissionless operator base that reachability dependency — on thousands of operators' firewalls no one controls — is arguably its biggest weakness, bigger than the one-time engineering cost. So QUIC needs a TCP fallback (as every HTTP/3 deployment ships), and its HOL/priority wins land only on UDP-reachable paths.

Three caveats apply to any separation:

It gives up the single-bearer streaming properties. One TCP bearer per peer is a single ordered, reliable, congestion-controlled stream with unified backpressure and one telemetry point. Splitting the traffic surrenders cross-protocol ordering (usually harmless — the mini-protocols are independent — but no longer guaranteed), the single congestion-control loop (now two competing cwnds, or CC we implement ourselves over UDP), unified backpressure (two domains), and shared-fate simplicity (partial failure — urgent up, bulk down, or vice versa — is a new state to handle). The mux's per-protocol fairness and telemetry must be replicated on the second channel.
It escapes the delivery-stall coupling, not interface contention. Separation removes coupling #3; it does not remove inter-connection NIC competition (gap 5) — the urgent flow still shares the interface with the bulk flow. Being tiny it wins its share easily unless the NIC is truly saturated, but "separate connection" is not "separate bandwidth".
It surrenders the mux's bandwidth arbitration, inverting Praos-over-Leios. On one bearer the mux scheduler (WFQ) can weight Praos above Leios; split the bulk flow off and the egress bottleneck arbitrates the two connections by priority-blind per-flow TCP fairness, so the single Leios closure fetch becomes co-equal (~50%) with the entire Praos suite on the main bearer — worse under fan-out, where per-flow fairness rewards Leios's many concurrent closure-serving flows. BBR compounds it (its design is to not defer — the opposite of the requirement). Restoring the priority needs an external scheduler: local egress qdisc priority-classing (moving the guarantee from the in-process mux to per-node OS config — gaps 13/16), or a scavenger CC (LEDBAT) that yields but sacrifices Leios throughput. QUIC is the exception — one cwnd plus internal stream priority keeps Praos>Leios without splitting into competing flows.

Once separated, the bulk flow's congestion control becomes a latency lever for the urgent flow. They now share the bottleneck as independent flows, so the bulk controller's queueing discipline is felt by the urgent flow. A CUBIC bulk flow fills a deep buffer until loss (bufferbloat), inflating the standing queue the vote flow must traverse; BBRv3 caps its inflight near one BDP and periodically drains, holding a shallower queue — so BBR bulk is a better latency neighbour than CUBIC. Two precisions keep this honest: BBR bounds the queue by model (inflight ≈ BDP, windowed min-RTT filter, periodic ProbeRTT), not by throttling on observed RTT rise, so it reduces rather than eliminates the standing queue; and its help scales with the BBR-managed share of the bottleneck load — it cannot undo a third flow's bufferbloat, so scheduling (fq/AQM/priority-classing the vote flow) stays the robust isolation tool, BBR complementary to it. That share is favourable here, though: EB closures are the dominant bulk on the connection and are our own egress, so putting them on BBR manages most of what would bloat the buffer — the latency benefit largely lands even without AQM tuning. This is the flip side of the shared-bearer note in the intra-connection couplings: CC is orthogonal to head-of-line blocking on one socket, but a real lever once the flows are separated.

Net: for votes — tiny, signed, loss-tolerant, deadline-critical — a separate channel is well-motivated, and QUIC streams (or, more conservatively, a second TCP bearer) are the sound realizations; raw UDP is tempting for its zero-HoL simplicity but reintroduces the congestion-control and operational problems that TCP and QUIC give for free. It is the last resort in the priority stack — warranted only where the bulk bearer's loss-recovery stall is what actually threatens the deadline.

Uh oh!

leios testnet

DeltaQ report validation and driving Leios parameters

Testnet trace collection

Validation vs. parameter-driving — different use cases

What a good testnet campaign looks like

Where this connects to work already in flight

Bottom line, and the rest of this document

Production cost of the DeltaQ machinery

What the branch adds on top of main's existing cost

If Shape B (per-protocol DeltaQ) were implemented

The tx-submission direction inversion

Mitigation strategies

What this means for the two Leios frames

The mitigation-vs-safety interaction

Measurement validity under load

Adaptive Leios via runtime measurement

What the reader view actually enables

Static vs. adaptive parameter setting — the category shift

Concrete adaptive behaviours it enables

Detecting mesh-wide stress

Traces for offline study

Why this doesn't come for free

Why it's specifically interesting for Leios

The short answer

Fully blended diffusion

Diagnosing pre-diffusion: transport vs. outcome

The two signals

The failure-mode diagnostic table

Baseline is a prerequisite

Investigating fully-blended diffusion when 1-hop fails

What "fully blended diffusion" actually requires empirically

What DeltaQ traces give you directly

What only needs a topology snapshot (cheaper than correlation)

What requires cross-node correlation

Measuring $F_{\text{full}\mid C}$ directly (Yves Hauser's §5.6 target)

Measuring churn-induced peer correlation and orphan-set geography

Measurement pre-requisites

Comparison across configurations

What requires inducing the scenario

Dynamics the report doesn't model, but DeltaQ traces expose

Cascade to livelock: bounding sustained-overload failure

Measurement recipes

Cross-references

The practical shape of a "blended-diffusion investigation" campaign

Two limits you can't get past

The short answer

Vote diffusion, head-of-line blocking, and Praos priority

Why votes need their own instrumentation

The intra-connection head-of-line couplings

Two-instance LeiosFetch: measure the mitigation

Composing with mux WFQ

Connection separation for latency-critical traffic

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally