Bus address claim: dynamic node_id assignment on shared-medium segments - #209
Merged
Conversation
Adds a protocol for claiming unique node_ids on bus-style network segments (CAN FD, ESP-NOW, RS-485) where multiple devices share a single interface. - Add AddressClaimEndpoint and AddressRefreshEndpoint with lease-based mechanics matching the existing seed router pattern - Add NodeBitmap ([u32; 8]) for O(1) per-frame validation of claimed node_ids in router process_frame - Add NodeClaim table with new const generic C on Router<I,R,N,S,C> (defaults to 0 for backwards compat) - Relax process_frame node_id check to accept any non-zero, non-CENTRAL node_id (bus-style support) - Add bitmap validation exception for wildcard port 0 (claim requests) - Relax EdgePort::set_state to allow changing node_id (bus devices start with a random candidate then switch to the granted id) - Add address_claim_handler() service - Add Profile trait methods: request_node_claim, refresh_node_claim, is_node_claimed - Add 4 e2e tests: claim+ping, two edges claiming, conflict detection, duplicate claim idempotency
Bridge upstream now starts as Active{net_id:0} like edge targets,
allowing the bridge to initiate contact with the root router without
waiting for an incoming frame first.
Pending downstream slots with net_id=0 were intercepting link-local frames (dst.network_id=0) meant for the upstream, causing DestinationLocal errors on bridges before seed assignment.
Previously only frames with both network_id=0 AND node_id=0 got src rewritten. Forwarded frames with src=(0, N, port) — e.g. from an upstream with link-local net_id — escaped with network_id=0, causing receivers to drop them as spoofed. Now any frame with src.network_id=0 gets the network_id rewritten to the outgoing interface's net_id. node_id is only rewritten if also 0 (locally originated).
The eusb RxWorker was unconditionally setting Active{net_id:0} on
every USB connection, overwriting a real net_id that was already
assigned (e.g. via seed routing before USB physically connected).
Now the handler skips the state update if the interface is already
Active, preventing the race where seed_assign sets net_id=3 and then
USB connect resets it to net_id=0.
On a bridge upstream, frames with various dst.network_ids pass through for routing to downstream interfaces. EdgeFrameProcessor was updating its net_id from every such frame, corrupting the upstream's identity. Now net_id is only discovered once (when None) from the first incoming frame. After liveness timeout, reset() clears it to allow re-discovery.
Tests the full ESP-NOW demo topology: Host ← Bridge ← Root with link-local upstream, seed routing, device discovery through the bridge, and upstream net_id stability after transit frames (regression test for EdgeFrameProcessor re-discovery bug).
Bus mock simulates a shared medium (ESP-NOW, CAN FD, RS-485) using tokio::sync::broadcast. Each BusTap can send frames to all other taps. Tests: - two_edges_on_shared_bus_claim_and_ping: two edges claim unique node_ids on the same net_id, router pings both - bus_claim_conflict_same_segment: two edges try to claim the same node_id on the same bus, second gets Conflict
The router allocated net_ids from a strictly monotonic counter that was never reused, so a long-running router with device churn would eventually exhaust the u16 space. Switch to lowest-free allocation that reclaims a net_id once its slot is removed or its seed-route tombstone clears, and make the tombstone grace period genuinely reserve the net_id until reuse is safe. - Replace the monotonic net_id_ctr with net_id_in_use + a lowest-free alloc_net_id (skips direct slots, seed routes incl. tombstones, and the upstream net_id — the last also fixes a latent bridge collision) - gc_seed_routes now tombstones expired active routes before removing them (adapted from jamesmunns#204 by @tommasoclini), the prerequisite for reclaiming a net_id - Anchor the tombstone grace to expiration + GRACE at every lease-expiry site (find, seed/node refresh, gc) so the reservation window starts when authority is lost; deregister keeps now + GRACE (no expiration exists) - register_interface runs gc_seed_routes first so cleared tombstones release their net_ids - Tests: reuse a freed direct net_id; tombstone reserves a seed net_id while a freed net_id is reused
tokio_serial::register_router gained the CC node-claim const generic in its own signature but did not forward it when delegating to tokio_cobs_stream::register_router, so any build enabling the tokio-serial transport failed with E0107 (7 of 8 generic arguments supplied). This broke `cargo clippy/doc --all-features`.
The router validated bus address claims with a single global NodeBitmap keyed only by node_id, but claims are per-(node_id, net_id). On a router with multiple bus segments this leaked across them: a node_id claimed on one bus validated frames bearing the same node_id on another, and GC clearing a tombstone could clear a bit still in use on a different bus or even the permanent CENTRAL/EDGE bits. request_node_claim also accepted reserved candidates (0/CENTRAL/EDGE/255). - Drop NodeBitmap; is_node_claimed now scans node_claims scoped to the frame's net_id, and treats an expired-but-not-GC'd claim as invalid so a quiet bus can't keep a stale node_id alive - request_node_claim rejects reserved node_ids with a new AddressClaimError::InvalidNodeId - gc_node_claims collapses to a single-pass retain_mut mirroring gc_seed_routes - Tests: claim is net_id-scoped (CENTRAL/EDGE always valid, no cross-bus leak); reserved node_ids are rejected
The refresh half of the claim protocol and its error paths had no test coverage. Add unit tests exercising refresh (lease extension + token rotation, TooSoon, BadRequest, UnknownNodeId) and the claim error paths (UnknownSource, Exhausted, duplicate-nonce idempotency, conflict). These run without a clock mock because the initial 30s lease is below MIN_SEED_REFRESH (62s), so a refresh is immediately permitted.
The bus address claim protocol had a server handler but no client API — callers had to hand-build the request and update interface state, and nothing refreshed a lease (so claims silently expired after 30s). Add bus_claim and bus_claim_refresh, mirroring bridge_seed_assign/refresh: bus_claim sends the link-local request and, on grant, sets the interface to Active with the granted address, returning a NodeClaimLease; bus_claim_refresh renews it. - New NodeClaimLease + ClaimClientError + bus_claim + bus_claim_refresh in services.rs - Refactor bus_edge_claims_and_pings to use bus_claim - New e2e test bus_edge_claims_then_refreshes covers the refresh endpoint, handler, and refresh_node_claim end-to-end (the previously untested half)
Seed routes and bus node_id claims had independent but near-identical lease lifecycles (active/tombstone state, expiry→tombstone GC, token-checked refresh with extend+rotate, grace anchored to expiration). Extract that lifecycle into a representation-agnostic LeaseKind + LeaseTable<K, X>, so it is written once and both tables reuse it. The key type K is the leased value: an assigned net_id (u16) for seed routes, a node_id (u8) for claims. This is also the seam for phone-number addressing (jamesmunns#145): the tree's parent→child allocation is the same lease lifecycle with K becoming an address range, so only K and the wire format change, not the machinery. - LeaseKind::{active,is_active,gc_retain,refresh}; LeaseTable::{gc, contains_key,by_key,get,get_mut,push} - seed_routes: LeaseTable<u16,u8,S> (extra=via_ident); node_claims: LeaseTable<u8,u64,C> (extra=nonce) - Unify refresh lookup to (key, scope) for both. Behaviour change: a seed refresh from the wrong source_net now returns UnknownNetId rather than BadRequest (the lease simply isn't found for that requester); test updated - No functional change otherwise; all 115 tests pass
The SinkEvent variants (SendTy/SendRaw/SendErr) mirror the sink methods send_ty/send_raw/send_err, so the shared prefix is intentional. This was the only clippy warning left on the branch; allow it locally for a clean build.
Mirror how seed routing is documented (book feature + notes design record): - notes/2026-06-01-bus-address-claim.md: design record — the central-arbiter choice and why it beats AppleTalk-style probing given a router, the one-router-per-segment assumption, the claim/lease/refresh/tombstone lifecycle, net_id reuse, the shared LeaseTable<K>, threat model, and the phone-number-addressing (jamesmunns#145) seam - book/_02 Major Concepts: note how Node IDs are claimed on a bus segment - book/_03 Feature Overview: new "Shared bus segment" topology under Connectivity-Now, and update the "Soon" RS-485/Radio bullet (Node ID addressing is now provided by the claim protocol)
Covers the validation drop path end-to-end at the process_frame level: a frame from an unclaimed node_id (to a non-zero port) on a bus is dropped, and the same frame routes normally once that node_id claims an address.
The duplicate-claim branch computed remaining lease time as `expiration - now`, which is safe today (gc tombstones expired actives before this branch is reached) but would panic on std if that invariant ever broke. saturating_duration_since is defensive and available on both std and embassy_time Instant.
After rebasing onto main, the bridge-discovery test no longer compiled: jamesmunns#207 made the COBS stream transport runtime-agnostic, replacing the tokio-specific CobsStreamRxWorker with the generic futures_io::RxWorker (constructed via `RxWorker::new(...).with_closer(...)` over a `.compat()` reader, driven by `run(&mut frame, &mut scratch)`). This mirrors the sibling e2e_bridge_seed test, which jamesmunns#207 already migrated. - Rewrite the manual pending-downstream worker setup to use futures_io::RxWorker instead of the removed CobsStreamRxWorker; pass the tokio duplex reader through `.compat()` and drop the boxed trait objects. - Use tokio_util::compat in the test directly; it is already an active dependency under the tokio-std feature (no new dev-dependency needed).
Bringing up a bridge's downstream interface required hand-rolling the transport: register the interface pending, wire a closer, build a futures_io::RxWorker over a .compat() reader with a placeholder net_id, allocate the COBS buffers, spawn both workers, and deregister on exit. That ~40-line dance was duplicated verbatim across two tests, and there was no downstream counterpart to register_bridge_upstream. Add tokio_cobs_stream::register_bridge_downstream, mirroring register_bridge_upstream: it creates the outgoing queue, registers the interface via register_interface_pending, spawns the RX/TX workers, and returns the pending ident so the caller can drive bridge_seed_assign. The processor starts at net_id 0 and adopts the assigned net_id once the lease lands. - Add register_bridge_downstream + BridgeDownstreamRegistrationError. - Refactor e2e_bridge_discovery and e2e_bridge_seed onto it, dropping the manual worker setup (and the direct futures_io::RxWorker / tokio_util usage it required).
Correctness (red-green, tests in bus_claim_validation.rs): - Claim validation now applies only to frames *originating* on the arrival segment (src.network_id == net_id after the zero-rewrite), not to transit frames. Previously a bus-claimed node_id (3..254) routed through a second Router was dropped, breaking multi-hop routing of bus-originated traffic. Test: transit_frame_from_foreign_net_is_not_dropped. - deregister_interface now purges node_id claims scoped to the removed interface's net_id. They were lingering until lease expiry and, on a reused net_id, would validate frames / block re-claims on the new bus. Test: claims_are_purged_when_interface_deregistered (added LeaseTable::drop_scope). Cleanups: - Fix is_node_claimed default-impl doc (it accepts only CENTRAL/EDGE, not "all node_ids" as the comment claimed). - Rename the shared lease constants INITIAL/MAX_SEED_ASSIGN_TIMEOUT and MIN_SEED_REFRESH to INITIAL_LEASE_SECS / MAX_LEASE_SECS / MIN_REFRESH_SECS now that LeaseTable shares them between seed routes and node claims. - Document that LeaseKind::refresh checks the token before expiry on purpose (a wrong token learns nothing about lease existence/expiry). - Rename the misleading e2e_bus_claim "two_edges...same bus" test to reflect that it uses two separate point-to-point links (different net_ids); the real shared-medium contention test lives in e2e_bus_segment. Deferred: skipping pending (net_id=0) slots in the port-255 broadcast loops (a marginal consistency guard) — left out rather than ship a behavior change without a dedicated broadcast-routing test.
jamesmunns#204 (router_gc_fix, by @tommasoclini) is an open PR, not a merged upstream fix, and it only tombstones expired seed routes (freeing table slots) — it does not reclaim net_id values, since the monotonic counter remains. State that accurately: the gc fix is adapted/generalized here, and this work supersedes jamesmunns#204 by also reusing net_ids via lowest-free allocation.
- Reframe the design note's "Open questions": keep only candidate-selection/ retry (the one genuine open question); move router redundancy and routerless/ mesh modes to a "Non-goals" section, since both break the strict-tree, one-arbiter-per-segment invariant this design relies on, and fix the dangling cross-reference in the assumption paragraph. - Update the Validation section: validation applies only to frames originating on the arrival segment (src.network_id == net_id); transit frames are exempt so bus node_ids route across the tree (matches the implementation). - Update Expiry/tombstones: a deregistered bus interface drops its claims outright (not tombstones), since its net_id may be reused; note this on Router::deregister_interface too. - Note the transit-exemption bypass in the threat model. - Clean up the shared-bus ASCII diagram in the feature-overview book section.
The claim protocol leaves candidate selection to the device, which meant every caller hand-rolled the conflict-retry loop. Provide it: `bus_claim_with_retry` walks a caller-supplied candidate iterator (a range like `3..=254` for sequential probing, or an RNG-driven iterator for randomized probing), calling `bus_claim` for each and advancing on `Conflict`. A grant or any other error stops immediately; exhausting the candidates returns the new `ClaimClientError::NoFreeCandidate`. The same nonce is reused across attempts so a lost response to a granted claim is recovered idempotently. - Add `bus_claim_with_retry` + `ClaimClientError::NoFreeCandidate`. - Tests: skip a taken candidate and claim the next; exhaust to NoFreeCandidate. - Docs: resolve the design note's candidate-selection open question (only the candidate ordering is now left to the device) and list the helper in the book.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ergot assigns node_ids by fixed role on point-to-point links (controller = 1, target = 2), but a shared-medium segment — ESP-NOW, CAN FD, RS-485, a simple radio — can carry many devices on one net_id, each needing a unique node_id negotiated at runtime. This adds a router-arbitrated address-claim protocol (DHCP-like, not AppleTalk-style probing, since an ergot tree always has exactly one router per segment): a device boots link-local, proposes a candidate node_id, and the segment's router grants it as a lease or reports a conflict so the device retries. The router reuses the seed-router lease lifecycle (lease + refresh + tombstone + GC), so node_ids are reclaimed automatically as devices come and go. It is a defense against accidental collisions on a trusted bus, not a security boundary (no authentication). See
notes/2026-06-01-bus-address-claim.mdfor the full design and threat model.Routerprofile:request_node_claim/refresh_node_claim/is_node_claimed, two well-known endpoints (address/claim,address/refresh), theServices::address_claim_handlerserver, andbus_claim/bus_claim_refreshclient helpers (mirroringbridge_seed_assign).LeaseTable<K, X>(active → tombstone → GC, token-checked refresh with extend + rotate).Kis the seam for phone-number addressing (Add phone number addressing proposal #145): a node_id today, an address range later.alloc_net_idnow hands out the lowest free net_id (skipping in-use slots, seed routes, tombstones, and the upstream), so a long-running router under device churn no longer climbs the u16 space to exhaustion.EdgeFrameProcessorre-discovering its net_id from transit frames (a bridge upstream's net_id now stays stable), plus link-local routing fixes (skip pending slots in routing, rewritesrc.network_id = 0regardless of node_id, start a bridge upstream link-local).register_bridge_downstream(the missing sibling ofregister_bridge_upstream) for bringing up a pending downstream interface, and stop the eusb transports clobbering an already-Activeinterface state on USB connect.Routerprofile gains an optionalconst C: usize = 0(max bus claims); existingRouter<I, R, N, S>usages and toolkit aliases compile unchanged, and theregister_routertransport helpers threadCthrough via inference.LeaseTable's GC. This goes further by replacing the monotonic net_id counter with lowest-free allocation, so net_id values are reclaimed too; fix router's gc function #204 alone frees slots but the counter still climbs to exhaustion under sustained churn.e2e_bus_segment), claim/idempotent/refresh logic (e2e_bus_claim), frame-validation incl. transit and deregister-purge (bus_claim_validation); plus a book section.Supersedes #204.