Skip to content

Decode advert node type as enum; add sensor type; drop empty packets - #49

Open
ajvpot wants to merge 1 commit into
mainfrom
ajvpot/advert-type-decode
Open

Decode advert node type as enum; add sensor type; drop empty packets#49
ajvpot wants to merge 1 commit into
mainfrom
ajvpot/advert-type-decode

Conversation

@ajvpot

@ajvpot ajvpot commented Jun 19, 2026

Copy link
Copy Markdown
Owner

Advert node type: enum, not bit flags

The advert app_data flags byte (Core Protocol §2.8.3) uses split encoding: the low 4 bits are an integer node-type enum and the high 4 bits are independent presence flags.

bits 0-3 (0x0F): node type — 0=none, 1=chat/companion, 2=repeater, 3=room server, 4=sensor
bit 4 (0x10): has_location   bit 5 (0x20): feature1   bit 6 (0x40): feature2   bit 7 (0x80): has_name

The decode treated the type as independent bit flags (is_chat_node = flags & 0x01, is_repeater = flags & 0x02, is_room_server = flags & 0x03). Because a room server is type 3 (0b0011) it matched all three at once (shown as repeater + companion + room simultaneously), and a sensor (type 4) matched none (rendered as an untyped/unknown node).

Migration 008 corrects the derivation in the meshcore_adverts / meshcore_adverts_latest views, adds an is_sensor boolean and a node_type column, and carries neighbor_is_sensor onto the direct-neighbor graph. These are read-time view changes over already-stored data (appdata_flags / payload), so the fix applies retroactively to all history with no reingest or backfill. The presence-flag (has_*) decoding was already correct and is unchanged. Full Down restores the prior view/MV definitions.

The UI gains a sensor badge (node page, node cards, advert details), getNodeType is corrected, and is_sensor is threaded through the node/neighbor/search/streaming queries, types, and API docs. Grafana reads is_repeater/is_room_server straight from the view, so its node-type counts self-correct on the next refresh.

Ingest: drop empty packets

A growing share of upstream publishers (observer nodes, repeater tx-confirmations) emit an otherwise-valid JSON envelope with an empty/absent packet-bytes field. Stored, these decode to an empty payload and a degenerate packet_hash and carry no signal. enqueuePacket now drops zero-byte packets, counted and logged via droppedEmpty. (Confirmed by sampling the live feeds: the bytes are simply not present upstream — not a renamed field — so nothing decodable is lost.)

Validation

  • Full migration chain applied → rolled back to 7 → re-applied cleanly on a throwaway ClickHouse; new columns present on all three objects.
  • A synthetic room-server advert (flags 0x03) decodes to node_type=3, is_room_server=1 with is_chat_node=0, is_repeater=0 (previously chat+repeater+room all 1).
  • Decode logic cross-checked against real data: every node maps to exactly one type; no more chat∧repeater overlap.
  • tsc --noEmit + lint clean; go build + go test pass (incl. new empty-drop test).

🤖 Generated with Claude Code

The advert app_data flags byte (Core Protocol §2.8.3) uses split encoding:
the low 4 bits are an integer node-type enum (1=chat/companion, 2=repeater,
3=room server, 4=sensor) and the high 4 bits are independent presence flags.
The decode treated the type as independent bit flags, so a room server (type
3 = 0b0011) matched is_chat_node, is_repeater AND is_room_server at once, and
a sensor (type 4) matched none and rendered as an untyped node.

Migration 008 fixes the derivation in the meshcore_adverts / meshcore_adverts_latest
views (read-time only, retroactive over all history), adds is_sensor and a
node_type column, and carries neighbor_is_sensor onto the direct-neighbor graph.
The UI gains a sensor badge across the node page, node cards, and advert details,
and search/streaming/node queries expose is_sensor.

Ingest: drop envelopes whose decoded packet has zero bytes instead of storing
them. They decode to an empty payload and a degenerate packet_hash and carry no
signal; dropped rows are counted and logged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ajvpot
ajvpot force-pushed the ajvpot/advert-type-decode branch from 01a4b04 to 0265a90 Compare June 20, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant