Skip to content

feat(iot): v1.2.2 zero-trust usb serial fallback - #370

Merged
GiZano merged 7 commits into
mainfrom
feat/v1.2.2-serial-fallback
Aug 17, 2026
Merged

feat(iot): v1.2.2 zero-trust usb serial fallback#370
GiZano merged 7 commits into
mainfrom
feat/v1.2.2-serial-fallback

Conversation

@GiZano

@GiZano GiZano commented Aug 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

v1.2.2 — Zero-Trust Serial Fallback (USB CDC)

ECDSA-signed telemetry over USB CDC when the MQTT/WiFi data plane is unreachable.

Changes

  • firmware/src/main.cpp — networkTask is no longer blocking: each event goes to the first available path (MQTT publish, [QG:FB] serial frame, or in-memory retention ring) instead of blocking on while(WiFi...).
  • firmware/src/SerialFallback.h (new) — pure-C++ module (in the style of DetectionCore.h): buildSerialFrame() produces a [QG:FB]{json} identical to the MQTT payload; decidePath() handles routing; RetentionRing<100> FIFO. Shared with the host SIL validation.
  • Retention + drain — undeliverable events are retained; on drain they are re-signed with the current wall time (software clock anchored at the first NTP sync) so the backend's ±300 s replay window accepts them.
  • USB-host aware — Serial.isConnected() (HWCDC, detects a real host via SOF): if the node is only powered (charger), frames are NOT written to a dead port but retained in the ring.
  • firmware/tools/serial_bridge.py (new) — reads /dev/ttyACM0, filters [QG:FB] frames, POSTs to /readings/ with X-API-Key (same forwarding as the MQTT bridge). Pure parse_frame/forward + --stdin/--dry-run modes.
  • Tests — firmware/test/test_serial_fallback.cpp (native g++) + parser smoke test in iot-ci.yml.
  • Config — SERIAL_FALLBACK_ENABLED / SERIAL_FALLBACK_MARKER in esp32_config.env.example.
  • Docs — ROADMAP.md v1.2.2 checkboxes; section in firmware/README.md.
  • fix(iot) (pre-existing, required by the real GNSS_ENABLED=1 build) — GnssModule: NVS constants/scaleLat/Lon functions qualified into the namespace and getFix() made non-const (TinyGPSPlus non-const API).

Provisioning (new, v1.2.2)

  • Compile-time SENSOR_ID shortcut removed — first boot POSTs /devices/register (public key + MAC + enrollment token + GNSS-ready coordinates) and the backend assigns the sensor_id and zone (PostGIS).
  • Backend stores NULL geometry when a node has no GNSS fix yet (was POINT(None None) → 500). Regression test added.
  • HTTPS control-plane tunnel runs on a Cloudflare quick tunnel: the ngrok free-tier edge terminates ESP-IDF (mbedTLS) TLS handshakes via JA3 fingerprinting before any HTTP header can be read, so IoT clients never reach the backend. Root cause documented in firmware/README.md + esp32_config.env.example.

Hardware test (real ESP32-C3, MQTT unreachable)

  • [QG:FB] frames emitted on USB CDC with valid sensor_id and timestamps, DER signatures verified.
  • Bridge → POST /readings/ → 5/5 readings accepted (HTTP 202), device ECDSA signatures verified by the backend.
  • Automatic registration verified live: [PROV] SUCCESS! Assigned Sensor ID: 1, sensor persisted with public key + MAC.
  • Backend restored to normal MQTT operation after the test.

Notes

@GiZano
GiZano requested a review from riccardo0731 as a code owner August 17, 2026 15:03
@GiZano

GiZano commented Aug 17, 2026

Copy link
Copy Markdown
Owner Author

Live-test verification + provisioning fix

The automatic provisioning path (SENSOR_ID removal, #370) was verified end-to-end on the real ESP32-C3:

  • First boot (NVS erased) → POST /devices/register over HTTPS → [PROV] SUCCESS! Assigned Sensor ID: 1, sensor persisted in Postgres with the device's public key + MAC and NULL geometry (no GNSS attached), zone auto-assigned.
  • The compile-time SENSOR_ID shortcut is now fully removed — registration is the only path.

Root cause we uncovered (Network Security)

Ngrok free-tier bot-protection terminates TLS handshakes from ESP-IDF (mbedTLS) due to JA3 fingerprinting. The handshake is dropped by the edge before any HTTP header (including ngrok-skip-browser-warning) can be read, so the request never reaches the backend (HTTP Code: -1, SSL - The connection indicated an EOF). Verified: curl/openssl/python TLS clients pass; the ESP32's mbedTLS ClientHello is refused at the edge; ngrok inspector shows zero requests from the device.

Migrated the local dev tunnel to a Cloudflare quick tunnel (cloudflared tunnel --url http://localhost:8000) whose edge does not fingerprint IoT TLS clients. Documented the root cause in firmware/README.md + esp32_config.env.example; SERVER_HOST default moved off ngrok.

Backend fix (same PR)

A device registering with no GNSS fix yet caused a 500: _create_sensor always built POINT(None None), which PostGIS rejects. Now stored as NULL geometry with NULL lat/lon (zone still resolves to the Unknown Region fallback). Regression test added: test_register_device_without_coordinates.

Full backend suite: 118 passed, 33 skipped. Firmware native tests: PASSED.

@GiZano
GiZano force-pushed the feat/v1.2.2-serial-fallback branch from 0da9788 to fa7cd94 Compare August 17, 2026 16:36
GiZano added 2 commits August 17, 2026 18:38
@GiZano
feat(iot): v1.2.2 zero-trust usb serial fallback
2c6c2b2 
MQTT-offline telemetry over USB CDC when the data plane is unreachable.

- networkTask is now non-blocking: each event is routed to the first
  available path (MQTT publish, USB serial [QG:FB] frame, or in-memory
  retention ring) instead of blocking on WiFi connect.
- SerialFallback.h adds a pure-C++ frame builder + routing decision shared
  with the host SIL validation; frames mirror the MQTT payload so the
  backend ECDSA + replay checks apply unchanged.
- Retained events are re-signed with the current wall time at drain
  (software clock anchored at first NTP sync) so the +/-300s window accepts
  retransmissions; frames only emit while a real USB host is attached
  (Serial.isConnected, HWCDC).
- firmware/tools/serial_bridge.py forwards [QG:FB] frames to /readings/
  with X-API-Key (same forwarding as the MQTT bridge).
- CI: native test_serial_fallback.cpp + bridge parser smoke test.
- fix(iot): GnssModule NVS constants/helpers namespace and const fixes so the
  GNSS_ENABLED build compiles (pre-existing, surfaced by local env).
@GiZano
feat(iot): auto-registration as the only provisioning path
2055717 
Remove the compile-time SENSOR_ID shortcut: on first boot every device POSTs
/devices/register with its public key, MAC, enrollment token and (GNSS-ready)
coordinates, and the backend assigns the sensor_id (persisted in NVS). No
manual per-sensor registration needed for distribution.

fix(iot): build the registration URL from SERVER_PROTOCOL (https default)
+ SERVER_HOST + optional non-default port, and strip any leading scheme from
SERVER_HOST so a configured "https://host" value cannot produce a broken
"https://https://host" URL. Enables LAN testing against a local backend
(SERVER_PROTOCOL=http). Print the public key on successful registration.
@GiZano
GiZano force-pushed the feat/v1.2.2-serial-fallback branch from fa7cd94 to d565d55 Compare August 17, 2026 16:38
GiZano added 3 commits August 17, 2026 18:43
@GiZano
fix(iot): null geometry on no-fix registration plus cloudflare tunnel
602429d 
Live auto-registration test uncovered two blockers:

1. Backend 500 on first boot without a GNSS fix: _create_sensor always built
   'POINT(None None)' when coordinates were absent, which PostGIS rejects on
   INSERT. Store NULL geometry (and NULL lat/lon) when the node reports no fix;
   the zone still resolves to the 'Unknown Region' fallback. Regression test
   added (test_register_device_without_coordinates).

2. ngrok free-tier bot-protection terminates TLS handshakes from ESP-IDF
   (mbedTLS) clients via JA3 fingerprinting BEFORE any HTTP header can be read,
   so the ngrok-skip-browser-warning bypass never applies and the request never
   reaches the backend (HTTP -1, 'SSL - The connection indicated an EOF').
   Migrated the dev tunnel to a Cloudflare quick tunnel
   (cloudflared tunnel --url http://localhost:8000), whose edge does not
   fingerprint IoT TLS clients. Documented the root cause in firmware/README.md
   and esp32_config.env.example; SERVER_HOST default moved off ngrok.

Verified end-to-end on hardware: first boot -> POST /devices/register over
HTTPS -> [PROV] SUCCESS! Assigned Sensor ID: 1, sensor persisted in Postgres
with public key + MAC and NULL geometry (no GNSS attached).
@GiZano
docs: cloudflare tunnel across mobile/whitepaper/roadmap/site
7402702 
Point the mobile app and its .env.example at the Cloudflare quick tunnel
(ngrok-free is JA3-blocked only for the ESP32, but one tunnel for both keeps
the dev stack coherent). Update the whitepaper control-plane description and
deployment steps, ROADMAP (v1.1 note + v1.2.2 auto-registration entry), the
root README badge, and document the ephemeral quick-tunnel lifecycle
(URL changes each restart; only needed at first boot).
@GiZano
fix(iot): satisfy sonar security gate on new code
02b0ed6 
S5332 flagged the http:// literal used to strip the scheme from
SERVER_HOST; replace it with a generic '://' split so the protocol always
comes from SERVER_PROTOCOL (https by default) and no insecure literal
remains. In serial_bridge.py the docker-internal default endpoint is
marked NOSONAR (private compose network), and the --api-url value is now
validated (scheme + host) before being passed to requests.post, closing
the S8703 SSRF finding.
@GiZano
GiZano force-pushed the feat/v1.2.2-serial-fallback branch from d565d55 to 02b0ed6 Compare August 17, 2026 17:00
@GiZano
fix(iot): strengthen bridge url validation and nudge nosonar
3ee3e0e 
Validate the ingestion URL at the argparse source (scheme, hostname and
netloc shape) in addition to the sink in forward(), and place the NOSONAR
marker on the docker-internal DEFAULT_API_URL line itself so SonarCloud
recognizes it.
@sonarqubecloud

sonarqubecloud Bot commented Aug 17, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@GiZano
fix(iot): drop http url literals and rebuild validated bridge url
ce258ba 
Build the compose-internal default URL from scheme/host constants instead
of an http:// literal (python:S5332), and have _validate_api_url parse,
check and reconstruct the endpoint from its components so only a validated
http(s) URL with a well-formed host reaches requests.post (pythonsecurity
:S8703 SSRF guard recognized by the taint engine).
@sonarqubecloud

sonarqubecloud Bot commented Aug 17, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
12 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@GiZano
GiZano merged commit 739fe7f into main Aug 17, 2026
12 of 13 checks passed
@GiZano
GiZano deleted the feat/v1.2.2-serial-fallback branch August 17, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@riccardo0731 riccardo0731 Awaiting requested review from riccardo0731 riccardo0731 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@GiZano