Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,9 +298,9 @@ Each `Playbook` (the field-guide entries in `catalog.py`, keyed by `detector_key

| Key | Signature (compressed) | Sev |
|---|---|---|
| `wired.bad_cable` | rx_errors delta rate > 10/min sustained or > 0.001% of packets; OR gigabit-capable peer negotiated at 10/100 (broken-pair downshift). Confounders: known-100Mbps device classes, counter age, unmanaged-switch hop | P2, P1 on uplink |
| `wired.bad_cable` | rx_errors delta rate > 10/min sustained or > 0.001% of packets; OR a broken-pair speed downshift on either arm — **rated** (gigabit-capable peer negotiated at 10/100) or **observed** (a port running below a speed it held itself in the last 7 days). The observed arm carries the check past gigabit: on a 2.5G/10G port a fall to 1000 is the same symptom, but 1000 is unremarkable for a 2.5G port carrying a 1G device, so only the port's *own* history proves the peer can do better. The ceiling is measured as **time-held, not row count**: `record_state_change` writes only on change, so a port that ran a week at 2500 carries one 2500 row dated a week ago, and the value in effect *entering* the window has to seed the timeline or the commonest shape of all — damaged once, renegotiated down once, sitting there since — reports nothing. (`prune` preserves that boundary row for exactly this read.) Rows are fetched **by window, never by row count** — a count-limited fetch drops the oldest first, which is where the seed lives, so the harder a link flapped the more certainly its ceiling would be truncated away. Confounders: known-100Mbps device classes, counter age, unmanaged-switch hop, and a peer newer than the speed it would be credited with — compared against **when that speed was actually held**, not against the window start, since the latter both rejects peers that demonstrably ran fast mid-window and silently disables the whole arm on any store younger than the lookback. Where several peers claim one port the **newest** wins, so a departed device's lingering entity cannot vouch for its replacement. An observed ceiling overrules the device-class list only when it **dominates** — held longer than the current speed — because a 10/100 camera that blipped to 1000 for twelve minutes during a cabling event is not a cable fault | P2, P1 on uplink |
| `wired.duplex_mismatch` | `full_duplex=false` on modern link | P2 |
| `wired.port_flapping` | ≥5 link transitions/10 min or ≥10/h from events; weight infra ports higher; correlate PoE draw 0 between flaps (reboot loop) | P2, P1 for AP/uplink |
| `wired.port_flapping` | ≥5 link transitions/10 min, ≥10/h, or ≥12/24 h; weight infra ports higher; correlate PoE draw 0 between flaps (reboot loop). The 24 h tier exists because the first two only see a link failing *fast* — a port dropping once or twice an hour around the clock never puts 5 transitions in any 10-minute window, and that slow-burn shape is what a marginal cable or a power-managed NIC actually produces. The title names the tightest tier that tripped, so a wide-window finding never renders as "0 transitions/10m" | P2, P1 for AP/uplink |
| `wired.uplink_saturation` | uplink bps > 80%/95% negotiated speed 5 min+ with rising tx_dropped; hour-of-day baseline first | P2 |
| `wired.poe_budget` | Σ poe_power > 80%/90% budget; `EVT_SW_PoeOverload` | P2/P1 |
| `wired.stp_loop` | `EVT_SW_StpPortBlocking`, stp_state churn | P1 active |
Expand Down
53 changes: 46 additions & 7 deletions netadmin/detect/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,9 +292,13 @@ def _channel_plan_spread_note(per_channel: Any) -> str:
),
"wired.bad_cable": Playbook(
signature="rx_errors delta rate > 10/min sustained or > 0.001% of packets; OR a "
"gigabit-capable peer negotiated at 10/100 (broken-pair downshift).",
confounders="Known 100 Mbps device classes; counter age (a stale cumulative counter); "
"an unmanaged-switch hop hiding the real port.",
"gigabit-capable peer negotiated at 10/100; OR a port running below a speed it "
"held itself in the last 7 days (broken-pair downshift, rated or observed).",
confounders="Known 100 Mbps device classes (overruled only by an observed ceiling that "
"outlasts the current speed, so a brief blip cannot condemn a 10/100 device); counter "
"age (a stale cumulative counter); an unmanaged-switch hop hiding the real port; a "
"wired peer newer than the speed it would be credited with (a faster device swapped "
"out for a slower one).",
fix_guidance="Reseat then replace the patch cable; re-test the run. On an uplink port "
"this is P1: the whole segment rides it.",
evidence_fields=(
Expand All @@ -303,6 +307,7 @@ def _channel_plan_spread_note(per_channel: Any) -> str:
EvidenceField("error_packet_fraction", "Errors, share of packets", percent=True),
EvidenceField("negotiated_speed", "Negotiated speed", "Mbps"),
EvidenceField("port_capable_speed", "Port's rated speed", "Mbps"),
EvidenceField("observed_speed_max", "Speed this link has held", "Mbps"),
),
confounder_notes={
"coverage_gated": _coverage_note,
Expand All @@ -328,6 +333,22 @@ def _channel_plan_spread_note(per_channel: Any) -> str:
if ev.get("negotiated_speed") is not None
else None
),
"peer_predates_observed_speed": lambda ev: (
"Peer age checked: the wired device on this port was already here before the "
f"link last held {_n(ev.get('observed_speed_max'), 0)} Mbps, so that speed is "
"this device's own history — not a faster machine that used to sit here."
if ev.get("observed_speed_max") is not None
else None
),
"observed_speed_regression": lambda ev: (
f"Measured against this link's own history: it has held "
f"{_n(ev.get('observed_speed_max'), 0)} Mbps recently and is now at "
f"{_n(ev.get('negotiated_speed'), 0)} Mbps, so the peer is provably capable of "
"the higher speed and something on the run is holding it back. A device that "
"simply cannot go faster would never have linked faster."
if ev.get("observed_speed_max") is not None
else None
),
},
),
"wired.duplex_mismatch": Playbook(
Expand All @@ -354,26 +375,44 @@ def _channel_plan_spread_note(per_channel: Any) -> str:
},
),
"wired.port_flapping": Playbook(
signature="≥5 link transitions/10 min or ≥10/h from events; infra ports weighted higher; "
signature="≥5 link transitions/10 min, ≥10/h, or ≥12/24 h; infra ports weighted higher; "
"PoE draw dropping to 0 between flaps signals a reboot loop.",
confounders="A laptop docking/undocking; scheduled device reboots.",
fix_guidance="Reseat cable/SFP; on a PoE reboot loop check the PoE budget and power-"
"cycle the port; replace the cable if errors persist.",
"cycle the port; replace the cable if errors persist. A port that only trips the 24 h "
"tier — dropping steadily around the clock rather than in bursts — is more often the "
"device end than the run: check NIC/adapter power management before re-cabling.",
evidence_fields=(
EvidenceField("transitions_short", "Transitions, short window"),
EvidenceField("window_short_s", "Short window", duration=True),
EvidenceField("transitions_long", "Transitions, long window"),
EvidenceField("window_long_s", "Long window", duration=True),
EvidenceField("transitions_sustained", "Transitions, sustained window"),
EvidenceField("window_sustained_s", "Sustained window", duration=True),
EvidenceField("poe_reboot_loop", "PoE reboot loop"),
EvidenceField("poe_min_w", "PoE draw, min", "W"),
EvidenceField("poe_max_w", "PoE draw, max", "W"),
),
confounder_notes={
"coverage_gated": _coverage_note,
# The sustained clause is guarded because issues predating that tier
# carry no such evidence: a resolved port_flapping issue never gets
# its evidence refreshed, so an unguarded f-string renders the old
# shape as "unknown in unknown" forever. Same contract as every other
# note here -- a missing key falls back, never fabricates.
"sustained_transition_count": lambda ev: (
f"Sustained, not a blip: {_n(ev.get('transitions_short'), 0)} transitions in the "
f"last {_dur(ev.get('window_short_s'))} ({_n(ev.get('transitions_long'), 0)} in "
f"{_dur(ev.get('window_long_s'))})."
f"last {_dur(ev.get('window_short_s'))}, {_n(ev.get('transitions_long'), 0)} in "
f"{_dur(ev.get('window_long_s'))}"
+ (
f", {_n(ev.get('transitions_sustained'), 0)} in "
f"{_dur(ev.get('window_sustained_s'))}. A link that drops steadily all day "
"trips the widest window even when no single burst is fast enough for the "
"others."
if ev.get("transitions_sustained") is not None
and ev.get("window_sustained_s") is not None
else "."
)
),
# The evidence, not the confounder key, carries the verdict: this key only
# means PoE data existed to check, not that a reboot loop was confirmed
Expand Down
Loading
Loading