Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions doc/design/pq-transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,8 +319,9 @@ qbit reports each v2 connection's negotiation status as one of these values:
- <!--pq:value-->`off` when the switch is off, when a responder sheds its offer
under load, or after a local key generation, key check or encapsulation fault
that left the connection on plain v2;
- <!--pq:value-->`fallback` for an outbound connection that runs plain v2
because of earlier failures; inbound connections never fall back.
- <!--pq:value-->`fallback` for an automatic outbound connection that runs
plain v2 because of earlier failures; manual and inbound connections never
fall back.

There is no failed status. A connection that closes on any failure above or
in key confirmation, local faults included, reports `pending` until it is
Expand All @@ -342,7 +343,15 @@ removed.
- **Downgrade.** Stripping an offer or an accept means forging packets under
the ECDH keys, so it takes an active attacker who breaks ECDH in real time
or sits in the middle, and an unauthenticated session can't stop the latter
anyway. A passive recorder can't downgrade a connection.
anyway. A passive recorder can't downgrade a connection. qbit's fallback
policy is weaker: an attacker on the network path, without any keys, can
make hybrid handshakes with an endpoint fail after the switch, by damaging
the peer's key confirmation in transit, resetting the connection or
dropping its traffic. Three such failures in a row put an automatic
outbound peer in the fallback set, so that new connections to it use ECDH
keys only, for 1 hour, then 4, then 24, and the attacker can repeat this.
Manual peers (<!--pq:rpc-->`addnode`, <!--pq:opt-->`-connect`) are exempt:
their failures never count, and they always negotiate.
- **Transcript binding.** Both keys, both contents and the network magic feed
the derivation, so two sides that saw different negotiation bytes derive
different keys and fail the confirmation instead of continuing.
Expand Down
3 changes: 2 additions & 1 deletion doc/integration/rpc-delta-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,13 +274,14 @@ Important result fields:
- `instance_id` and `since`: the process start the counters count from
- `handshakes.inbound` and `handshakes.outbound`: event counters, such as `switched`, `legacy_peer`, `first_packet_failed` and, inbound only, `shed`
- `load_shedding`: `active`, `threshold_per_s` and `since`
- `fallback_set` and `failure_streaks`: outbound endpoints that fall back to plain v2, or are on their way there
- `fallback_set` and `failure_streaks`: automatic outbound endpoints that fall back to plain v2, or are on their way there; manual connections (`addnode`, `-connect`) never fall back
- `recent_failures.inbound` and `recent_failures.outbound`: the most recent non-success outcomes, with sequence numbers

Integrator notes:

- Every field is always present. The counters, rings and endpoint history are kept in memory only and start over when `instance_id` changes.
- Counters count events, not exclusive outcomes: a switch whose key confirmation then fails counts `switched` and `first_packet_failed`.
- A `fallback_set` entry's `expires` is the latest its window ends: the window also ends once `window_seconds` have passed on the node's monotonic clock, so after a clock step back it can end earlier.
- Endpoints contain peer addresses; treat the output as sensitive.

## Shared RPC Names With qbit-Specific Semantics
Expand Down
10 changes: 6 additions & 4 deletions doc/release-notes-240.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,12 @@ handshake outcomes, load shedding and the fallback set. `getpeerinfo` gains
<!--pq:field-->`transport_pq` and <!--pq:field-->`transport_pq_status`,
`getnetworkinfo` gains <!--pq:field-->`connections_pq`, `qbit-cli -netinfo`
marks hybrid peers <!--pq:value-->`2p`, and the log states the setting at
startup in a <!--pq:log-->`v2 pq: … arith=… keccak=…` line. Outbound
connections to an address whose hybrid handshakes keep failing fall back to
plain v2 for a while, and a node under an inbound connection flood stops
offering hybrid keys until it subsides.
startup in a <!--pq:log-->`v2 pq: … arith=… keccak=…` line. Automatic
outbound connections to an address whose hybrid handshakes keep failing fall
back to plain v2 for a while. Someone on the network path can cause such
failures, so peers added with `addnode` or `-connect` never fall back. A node
under an inbound connection flood stops offering hybrid keys until it
subsides.

See the [operator guide](user/network/pq-transport.md) to try it, check it and
troubleshoot it, and the [specification](design/pq-transport.md). (#240)
55 changes: 34 additions & 21 deletions doc/user/network/pq-transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ verified, and <!--pq:field-->`transport_pq_status`:
| <!--pq:value-->`pending` | Still negotiating. A connection that is closing after a hybrid failure also stays `pending` until it is removed: there is no failed status, and the failure is counted in `getpqtransportinfo`. A handshake that doesn't finish within 60 seconds (`-peertimeout`) is closed. |
| <!--pq:value-->`hybrid` | The session keys are hybrid. |
| <!--pq:value-->`legacy_peer` | The peer's handshake carried no hybrid offer or accept, so the connection keeps the ECDH keys, as before. Peers running v1.0.0, or with the feature off, show this. |
| <!--pq:value-->`fallback` | Plain v2 to an address in the fallback set (outbound only). |
| <!--pq:value-->`fallback` | Plain v2 to an address in the fallback set (automatic outbound connections only). |
| <!--pq:value-->`off` | Plain v2 because the feature is off on this node, the offer was shed under load, or a local ML-KEM fault happened before anything was committed. |
| <!--pq:value-->`v1` | An unencrypted v1 connection. |

Expand All @@ -213,18 +213,30 @@ peers show `off` and `connections_pq` is 0.

## Fallback and Load Shedding

**Fallback.** If hybrid handshakes with the same outbound address fail three
times in a row, the node stops trying hybrid keys with that address for an hour,
and connects to it with plain v2 instead, so that a broken peer or a middlebox
that mangles the handshake can't keep the connection from working. The second
time the address falls back it stays out for 4 hours, and from then on for 24
hours. Only failures the peer side can cause count:
<!--pq:value-->`malformed_record`, <!--pq:value-->`first_packet_failed` and
<!--pq:value-->`closed_after_switch`. An address that completed a hybrid
handshake since startup never falls back, inbound connections never fall back,
and a restart forgets the whole history. `getpqtransportinfo` lists the
addresses in <!--pq:field-->`fallback_set`, with <!--pq:field-->`expires`, and
those on their way there in <!--pq:field-->`failure_streaks`.
**Fallback.** If hybrid handshakes of automatic outbound connections with the
same address fail three times in a row, the node stops trying hybrid keys with
that address for an hour, and connects to it with plain v2 instead, so that a
broken peer or a middlebox that mangles the handshake can't keep the connection
from working. The second time the address falls back it stays out for 4 hours,
and from then on for 24 hours. Each window is also measured on the node's
monotonic clock, so setting the clock back can't lengthen it. Only failures the
peer side can cause count: <!--pq:value-->`malformed_record`,
<!--pq:value-->`first_packet_failed` and <!--pq:value-->`closed_after_switch`.
An address that completed a hybrid handshake since startup never falls back
(the node remembers the 1,000 that did so most recently), inbound connections
never fall back, and a restart forgets the whole history. `getpqtransportinfo`
lists the addresses in <!--pq:field-->`fallback_set`, with
<!--pq:field-->`expires`, and those on their way there in
<!--pq:field-->`failure_streaks`.

**Manual peers never fall back.** Someone on the network path between you and
a peer can cause those failures without breaking any keys, by damaging or
dropping packets, and so keep an automatic peer on ECDH keys. Connections you
make yourself, with <!--pq:rpc-->`addnode` or <!--pq:opt-->`-connect` (also
<!--pq:opt-->`-addnode`), always negotiate hybrid keys, even to an address in
the fallback set, and their failures never count toward it; they are still
counted, listed in `recent_failures` and logged. If hybrid keys with particular
peers matter to you, add them with `addnode`.

**Load shedding.** An offer costs the responding node an ML-KEM key generation,
and a decapsulation when the peer accepts. If more inbound v2 connections than
Expand Down Expand Up @@ -255,8 +267,9 @@ It does not:
`session_id` out of band detects that.
- **Protect traffic to v1.0.0 nodes,** or to any peer that has the feature off:
those connections use ECDH keys (v2) or no encryption (v1), as before. The
fallback, load shedding and local faults also leave some connections on ECDH
keys; `transport_pq` reports which.
fallback, which someone on the network path can trigger for automatic peers,
load shedding and local faults also leave some connections on ECDH keys;
`transport_pq` reports which.
- **Hide where transactions come from.** Your peers see which transactions you
announce, and when, as before, which can reveal that one started at your node.
- **Hide that you use it.** Hybrid sessions can be told apart from plain v2 by
Expand Down Expand Up @@ -301,13 +314,13 @@ details such as `role=`, `peer=` and the ML-KEM-1024 backends; counters are in
| <!--pq:log-->`was given 0 or a non-numeric value, so it was read as` | A double negative switched the feature on. | `-nov2pqtransport=0` or `nov2pqtransport=true`. | Use `v2pqtransport=0` to switch it off. |
| <!--pq:field-->`legacy_peer` (counter and status) | The peer sent no hybrid offer or accept (reason <!--pq:value-->`no_features`), or contents qbit can't parse (<!--pq:value-->`parse_error`). | The peer runs v1.0.0 or has the feature off. `parse_error`: another implementation or a bug. | Nothing for `no_features`; it is normal while the network upgrades. `parse_error` from many peers: open an issue. |
| <!--pq:log-->`v2 pq: load_shedding started threshold_per_s=…`, <!--pq:log-->`v2 pq: load_shedding stopped`, counter `shed` | Load shedding started or stopped. | A connection flood, or many peers reconnecting at once. | Nothing; those connections continue as plain v2. If it stays on, check where inbound connections come from (`qbit-cli -netinfo 4`), and consider firewall rules or `-maxconnections`. |
| <!--pq:log-->`v2 pq: malformed_record reason=…`, counter <!--pq:field-->`malformed_record` | The peer's offer (<!--pq:value-->`ek_length`, <!--pq:value-->`ek_modulus`) or accept (<!--pq:value-->`ct_length`) was malformed. The connection closes; outbound, it counts toward the fallback. | A non-conforming implementation, or tampering in transit. | One-off: nothing. From many peers: report it. |
| <!--pq:log-->`v2 pq: first_packet_failed reason=…`, counter `first_packet_failed` | The peer's key confirmation failed (<!--pq:value-->`length`, <!--pq:value-->`tag` or <!--pq:value-->`not_decoy`): the two sides derived different keys. The connection closes; outbound, it counts toward the fallback. | Tampering in transit, a peer bug, or this node's ML-KEM code (see `local_fault`). | One-off: nothing. From many peers: report it. |
| <!--pq:log-->`v2 pq: closed_after_switch cause=…`, counter <!--pq:field-->`closed_after_switch` | After this node switched to hybrid keys, the peer closed (<!--pq:value-->`eof`), reset (<!--pq:value-->`reset`) or timed out (<!--pq:value-->`timeout`) before its key confirmation arrived. Counts toward the fallback. | The peer restarted, the network dropped, or the peer's check of our confirmation failed. | One-off: nothing. Often with one peer: ask its operator for their side's log. |
| <!--pq:log-->`v2 pq: malformed_record reason=…`, counter <!--pq:field-->`malformed_record` | The peer's offer (<!--pq:value-->`ek_length`, <!--pq:value-->`ek_modulus`) or accept (<!--pq:value-->`ct_length`) was malformed. The connection closes; on an automatic outbound connection, it counts toward the fallback. | A non-conforming implementation, or tampering in transit. | One-off: nothing. From many peers: report it. |
| <!--pq:log-->`v2 pq: first_packet_failed reason=…`, counter `first_packet_failed` | The peer's key confirmation failed (<!--pq:value-->`length`, <!--pq:value-->`tag` or <!--pq:value-->`not_decoy`): the two sides derived different keys. The connection closes; on an automatic outbound connection, it counts toward the fallback. | Tampering in transit, a peer bug, or this node's ML-KEM code (see `local_fault`). | One-off: nothing. From many peers: report it. |
| <!--pq:log-->`v2 pq: closed_after_switch cause=…`, counter <!--pq:field-->`closed_after_switch` | After this node switched to hybrid keys, the peer closed (<!--pq:value-->`eof`), reset (<!--pq:value-->`reset`) or timed out (<!--pq:value-->`timeout`) before its key confirmation arrived. On an automatic connection, it counts toward the fallback. | The peer restarted, the network dropped, or the peer's check of our confirmation failed. | One-off: nothing. Often with one peer: ask its operator for their side's log. |
| <!--pq:log-->`v2 pq: internal_error reason=…`, counter <!--pq:field-->`internal_error` | A local ML-KEM or cipher-state error (<!--pq:value-->`keygen`, <!--pq:value-->`check_public_key`, <!--pq:value-->`encaps`, <!--pq:value-->`decaps`, <!--pq:value-->`cipher_state`). Peer data never causes it. | A fault in this node's ML-KEM code or its CPU. | Report it. Try <!--pq:opt-->`-mlkemportable`. If a peer can trigger it, use `SECURITY.md`. |
| <!--pq:log-->`v2 pq: fallback cause=… until=…`, counter <!--pq:field-->`fallback` | An outbound address failed three times in a row, and gets plain v2 until the time given (UTC). | See the failure it names. | Usually nothing: the window ends by itself, and a restart clears it. Many addresses: see `local_fault`. |
| <!--pq:log-->`v2 pq: local_fault` | Hybrid handshakes failed with 8 different outbound addresses and none succeeded since startup, so this node's own ML-KEM code may be at fault. | A miscompiled or misbehaving native ML-KEM path, or a broken CPU. | If the line names native code, restart with `-mlkemportable`, as it suggests. Report it either way. |
| <!--pq:field-->`abandoned` (inbound) | An initiator read our offer and closed before sending its version packet. | Port scanners, peers giving up, flaky links. | Nothing at low rates. Many, together with `shed`: a flood. |
| <!--pq:log-->`v2 pq: fallback cause=… until=…`, counter <!--pq:field-->`fallback` | An address failed three times in a row on automatic outbound connections, and gets plain v2 on them until the time given (UTC). | See the failure it names. | Usually nothing: the window ends by itself, and a restart clears it. To keep hybrid keys with a peer, add it with `addnode`. Many addresses: see `local_fault`. |
| <!--pq:log-->`v2 pq: local_fault` | Hybrid handshakes failed with 8 different outbound addresses, in ways this node's own ML-KEM code can cause (`ek_modulus`, `length`, `tag`, or `closed_after_switch` with `eof` or `reset`), and none succeeded since startup, so that code may be at fault. Failures only the peer can cause (`ek_length`, `not_decoy`) and `timeout` don't count. | A miscompiled or misbehaving native ML-KEM path, or a broken CPU. | If the line names native code, restart with `-mlkemportable`, as it suggests. Report it either way. |
| <!--pq:field-->`abandoned` (inbound) | This node offered, and the connection closed before the initiator's version packet authenticated, whichever side closed it. The reason is the close cause, or what failed (<!--pq:value-->`garbage_terminator`, <!--pq:value-->`version_length`, <!--pq:value-->`version_tag`); <!--pq:value-->`local` is this node's own close, such as an eviction. Closes because the network was switched off, or at shutdown, aren't counted. | Port scanners, peers giving up, flaky links; with `local`, evictions when the inbound slots are full. | Nothing at low rates. Many, together with `shed`: a flood. |
| `transport_pq_status` stays `pending` | The handshake is still running, or the connection is closing after a failure. | A slow link, or a failure counted above. | It ends within 60 seconds. Look for the peer's `peer_id` in `recent_failures`. |
| `transport_pq_status` is `off` while `enabled` is true | This connection is plain v2. | The offer was shed, or a local fault (`internal_error`) before anything was committed. | Check `shed` and `internal_error`. |

Expand Down
Loading
Loading