HiveTraffic night mode: schedule, traffic gate, config and UI

.github/workflows/ci.yml

@@ -72,14 +72,25 @@ jobs:
           /tmp/test_beehive_decode
       # The HiveTraffic measurement decoder is a two-repo wire contract: it has
       # to keep reading fw:2 documents from counters the OTA relay has not
-      # reached yet as well as the fw:3 ones current firmware emits. Both
-      # captured documents are asserted here.
+      # reached yet, as well as the fw:3 and fw:4 ones newer firmware emits.
+      # A captured document of each revision is asserted here.
       - name: Build and run bee counter wire tests
         run: |
           set -e
           g++ -std=gnu++17 -Wall -Wextra -Werror -I firmware/include \
               -o /tmp/test_bee_counter_wire test-data/test_bee_counter_wire.cpp
           /tmp/test_bee_counter_wire
+      # Night mode decides whether to blind a bee counter, and gets it wrong
+      # quietly in both directions — a lost day of foraging looks like bad
+      # weather, and a counter that never sleeps looks like nothing at all.
+      # Midnight wrap, an unsynced clock and the traffic gate are all pinned
+      # here rather than left to a season of field observation.
+      - name: Build and run night mode tests
+        run: |
+          set -e
+          g++ -std=gnu++17 -Wall -Wextra -Werror -I firmware/include \
+              -o /tmp/test_night_mode test-data/test_night_mode.cpp
+          /tmp/test_night_mode
 
   server-tests:
     runs-on: ubuntu-latest

docs/api.md

@@ -78,7 +78,7 @@ Health check. No authentication required. `version` is the backend server
 version (`SERVER_VERSION` in `server/config.py`).
 
 ```json
-{ "status": "ok", "version": "0.3.0" }
+{ "status": "ok", "version": "0.3.1" }
 ```
 
 ### `GET /api/v1/time`
@@ -377,10 +377,36 @@ Returns the current config for a device. A default config is created if none exi
   "tempco_source": "ambient",
   "tempco_ref_temp_c": 20.0,
   "scale1_tempco_kg_per_c": 0.0,
-  "scale2_tempco_kg_per_c": 0.0
+  "scale2_tempco_kg_per_c": 0.0,
+  "beecounter_night_mode_enabled": false,
+  "beecounter_night_start_minute": 1200,
+  "beecounter_night_end_minute": 360,
+  "beecounter_night_max_traffic": 0,
+  "timezone": ""
 }
 ```
 
+The `beecounter_night_*` fields configure **HiveTraffic night mode** — see
+[hivetraffic-bee-counter.md](hivetraffic-bee-counter.md#night-mode). Unlike the
+calibration fields below, the device applies them on **every** config fetch
+rather than only when `config_version` changes: there is no portal-side
+counterpart to protect, and turning the window off has to take effect on the
+next cycle rather than waiting for an unrelated edit to bump the version.
+
+| Field | Meaning |
+| --- | --- |
+| `beecounter_night_mode_enabled` | Master switch. `false` by default, so an existing device is unaffected until it is turned on. |
+| `beecounter_night_start_minute` / `beecounter_night_end_minute` | The window, in **local** minutes since midnight (0–1439). `start > end` wraps midnight, which the 20:00–06:00 default does. `start == end` is an **empty** window, not a 24-hour one — the firmware refuses it, so a single mis-set field cannot stop a counter for good. |
+| `beecounter_night_max_traffic` | Crossings (in + out) in the last upload cycle above which night mode is postponed to the next cycle. `0` disables the check. |
+| `timezone` | POSIX TZ string, e.g. `CET-1CEST,M3.5.0,M10.5.0/3`. Empty means UTC. |
+
+`timezone` is load-bearing rather than cosmetic. The device clock is UTC, so
+without it a window entered as 20:00 fires at 21:00 local in summer — discarding
+an hour of foraging on exactly the long evenings that have most of it. It is a
+POSIX string rather than an IANA name because the ESP32 carries no tz database;
+newlib parses this form directly, DST rules included. The dashboard offers
+presets for the common zones and a free-text field for anything else.
+
 The `tempco_*` fields drive backend load-cell temperature compensation. They are
 informational for the device (which keeps sending raw weights); the correction is
 applied in the backend on read. See
@@ -1115,7 +1141,7 @@ The backend auto-creates and updates the schema on startup.
 | `devices` | Device identity, claim status, per-device API key hash, display name, last seen, firmware version, and the owner-approved firmware version (accept-to-apply gate) |
 | `device_members` | Users with `owner`, `admin`, or `viewer` role per device |
 | `device_channels` | Display names for scale channel 1 and 2 |
-| `device_configs` | Send interval, offsets, calibration factors, config version |
+| `device_configs` | Send interval, offsets, calibration factors, config version, bee-counter night mode |
 | `measurements` | Measurement records, including power/acoustic/bee-counter columns and `raw_json` |
 | `firmware_releases` | Firmware versions available for OTA, with `target`, `crc32`, and `owner_user_id` (NULL = global/official; otherwise the owner the release is private to) |
 | `device_commands` | Pending, claimed, done, and failed commands, with the `attempts` counter used to recover abandoned claims |

docs/hivetraffic-bee-counter.md

@@ -60,6 +60,15 @@ the counter reboots and the following measurement read reports the new version.
 and pauses gate polling while writing flash, so every bee crossing during those
 minutes is lost. Relay at night or in poor flying weather.
 
+That advice still holds with [night mode](#night-mode) enabled, and the two do
+not collide: a suspended counter stays advertising and connectable, so the relay
+runs exactly as it would in daylight. The counter refuses a *new* suspension
+while a transfer is in progress, HiveHub does not suspend a counter while a
+relay is unfinished, and the post-OTA reboot clears any suspension anyway — the
+next cycle re-arms it if the window is still open. This is the main reason night
+mode is a sensing suspension rather than deep sleep, which would have made the
+recommended relay window the one time of day a counter is unreachable.
+
 ### Version gate
 
 The relay is refused with `409` unless the release is strictly newer than the
@@ -107,8 +116,8 @@ All HiveTraffic devices share one service/characteristic (overridable via
 The characteristic returns a compact JSON document — **totals only**:
 
 ```json
-{ "fw":3, "ver":"0.2.0", "uptime_s":1234, "status":15, "num_gates":24,
-  "mcps_healthy":3, "total_in":100, "total_out":95, "glitches":2 }
+{ "fw":4, "ver":"0.2.0", "uptime_s":1234, "status":15, "num_gates":24,
+  "mcps_healthy":3, "total_in":100, "total_out":95, "glitches":2, "idle_s":0 }
 ```
 
 `fw` is the wire-protocol revision; `ver` is the counter's own image version,
@@ -119,16 +128,21 @@ HiveHub reads it, fills a totals-only `beecnt::Snapshot`, and disconnects.
 The wire format is totals-only by design: no latch/reset command exists over
 BLE, so a missed connection can never lose counts.
 
-### Two wire revisions, both supported
+### Three wire revisions, all supported
 
 `firmware/include/bee_counter_wire.h` reads `fw` **first** and branches on it.
-Both revisions parse, and both produce the same record:
+Every revision parses, and they produce the same record:
+
+| | `fw:2` | `fw:3` | `fw:4` |
+| --- | --- | --- | --- |
+| Expander health field | `gates_healthy` | `mcps_healthy` | `mcps_healthy` |
+| `uptime_s` | 16-bit on the device, clamped at 65535 (18 h 12 min) | 32-bit | 32-bit |
+| `glitches` | 16-bit, pinned at 65535 | 32-bit, saturating | 32-bit, saturating |
+| `idle_s` + status bit `0x80` | — | — | night-mode countdown |
 
-| | `fw:2` | `fw:3` |
-| --- | --- | --- |
-| Expander health field | `gates_healthy` | `mcps_healthy` |
-| `uptime_s` | 16-bit on the device, clamped at 65535 (18 h 12 min) | 32-bit |
-| `glitches` | 16-bit, pinned at 65535 | 32-bit, saturating |
+`fw:4` is purely additive, so the parser read those documents correctly before
+it knew `idle_s` existed — it skips unknown keys. What it could not do is tell a
+*suspended* counter from a broken one, which is the whole reason for reading it.
 
 This is not politeness toward old firmware. **A counter keeps reporting `fw:2`
 until the OTA relay updates it, and the relay reads this very characteristic
@@ -157,6 +171,83 @@ The decoder is covered by `test-data/test_bee_counter_wire.cpp`, which parses a
 captured document of each revision and asserts they decode identically. It runs
 in CI on a plain host compiler — no ESP32 and no counter required.
 
+## Night mode
+
+A HiveTraffic counter's 48 IR emitters are its power budget: 24 series pairs
+behind 22 Ω ballast, all lit together for the settle+read window of every 5 ms
+poll — roughly 0.5–1.0 A peak and, at the pulsed sampler's ~35 % duty, an
+average an order of magnitude above the ~18 mA the ESP32-C6 and its three port
+expanders draw between them. On an off-grid hive it is the whole supply.
+
+European honey bees are diurnal. Flight requires light — they do not fly in
+darkness at any temperature — and stops below roughly 10 °C regardless, so
+overnight that draw buys nothing. Night mode parks the emitters.
+
+**It is off by default.** Turn it on per device under **Device & admin →
+Bee counter night mode**, or `PATCH` the config fields directly (see
+[api.md](api.md#get-apiv1devicesdevice_idconfig)).
+
+| Setting | Meaning |
+| --- | --- |
+| Enable night mode | Master switch |
+| Night starts / ends | **Local** wall-clock times. The window may cross midnight; setting both the same disables it rather than covering the whole day |
+| Timezone | POSIX TZ string. The device clock is UTC, so without one the window drifts an hour at each DST change |
+| Postpone above | Crossings in the last cycle above which night mode waits for the next one. 0 goes by the clock alone |
+
+### How it works
+
+The counter has no RTC, no NVS and no network, and this design deliberately does
+not give it any. HiveHub owns the schedule and tells the counter only a bounded
+**duration** — "stop sensing for N seconds" — re-armed once per upload cycle for
+as long as the window lasts:
+
+1. `bee_counter_client.cpp` reads the measurement characteristic as usual.
+2. `night_mode.h::decide()` weighs the local clock, the window, the traffic gate
+   and whether a firmware relay is unfinished.
+3. If it says suspend, a 5-byte `SET_IDLE` frame goes out **on the same
+   connection** — no extra scan, no extra connect.
+4. The counter reports `idle_s` and status bit `0x80` until the grant expires.
+
+Every path that is not certain returns "keep counting": no valid local time, no
+traffic baseline yet, a window whose ends are equal, a relay in flight. The
+worst outcome of a bug here is a counter that counts, which is what it did
+before the feature existed. The grant is deliberately short (two upload cycles,
+capped at an hour on both sides), so a HiveHub that dies mid-night cannot leave
+a counter blind — the deadline simply runs out.
+
+The traffic gate is the "not yet" rule: if more bees crossed in the last cycle
+than the threshold, night mode waits. It is evaluated from the totals of the
+*previous* cycle, held in RTC memory across HiveHub's own deep sleep, which for
+a threshold like "fewer than 100 crossings in the last ten minutes" is the
+intended reading. A hive with no baseline yet — first cycle after a reboot — is
+postponed rather than waved through: "I don't know how busy this is" is not "it
+is quiet".
+
+### Why not deep sleep
+
+Deep sleep saves the residual ~18 mA on top of what parking the emitters already
+saves, under 10 % of the total, and costs the measurement read (every night row
+would carry `bee_counter.ok=false`), the firmware relay path that is
+specifically recommended for night use, the ability to cancel a wrong schedule,
+the counter's RAM-held lifetime totals, and a truthful `uptime_s`. The ESP32-C6's
+deep-sleep timer also runs off a temperature-dependent internal RC oscillator:
+over an 8–12 h sleep in a hive that swings 10–25 °C, expect minutes of drift and
+up to ~30 minutes worst case. Re-arming a short grant against HiveHub's DS3231
+(±2 ppm) means nothing accumulates. See HiveTraffic's `docs/ble-mode.md` for the
+device-side reasoning.
+
+### Reading the data
+
+The counter's totals are frozen while suspended, so the differenced interval
+across the window is a genuine zero rather than a gap — the same value a quiet
+night produces anyway. What `idle_s` adds is the ability to tell that zero from
+a counter whose emitter FETs have died, which otherwise produces an identical
+row. It is stored on every reading as `hives[].bee_counter.idle_s`.
+
+Counters running firmware older than `fw:4` have no control characteristic. The
+write is skipped and they keep counting; the OTA relay will bring them up to a
+firmware that can be suspended in the normal course of things.
+
 ## Intervals are differenced server-side
 
 The wire format carries only the **monotonic lifetime totals**. The backend

firmware/include/bee_counter_client.h

@@ -57,6 +57,11 @@ struct Snapshot {
     uint8_t  mcps_healthy     = 0;
     uint32_t total_in         = 0;      // lifetime totals — never reset by us
     uint32_t total_out        = 0;
+    // Seconds of night-mode suspension still to run on the counter ("idle_s",
+    // wire revision 4+); 0 while it is counting, and 0 for any counter too old
+    // to report it. Non-zero says the flat interval that follows is deliberate
+    // — we asked for it — rather than a failed emitter bank.
+    uint32_t idle_s           = 0;
     // 32-bit as of revision 3, saturating on the device rather than wrapping.
     uint32_t glitch_count     = 0;
 };

firmware/include/bee_counter_wire.h

@@ -19,6 +19,12 @@
 //            clamped at 65535 / pinned at 65535 respectively.
 //   fw >= 3  the same count is named "mcps_healthy"; uptime_s and glitches are
 //            32-bit.
+//   fw >= 4  adds "idle_s" and status bit 0x80: the counter can be told to stop
+//            sensing for a bounded period (night mode) and now says when it
+//            has. Purely additive, so this parser would have read a v4 document
+//            correctly before it knew the field existed — it skips unknown
+//            keys. What it could not do is tell a suspended counter from a
+//            broken one, which is the point of reading it.
 //
 // Both must keep working, and not as a courtesy: a counter in the field reports
 // fw:2 until it is updated over the air, and HiveHub's OTA relay has to read
@@ -35,8 +41,8 @@
 // Why a hand-written reader rather than ArduinoJson
 // ---------------------------------------------------------------------------
 // The document is generated by a single snprintf in HiveTraffic's
-// measurement_json.h: a flat object of nine fixed keys, no nesting, no arrays,
-// no escapes, ~171 bytes at its widest. A full JSON parser buys nothing for
+// measurement_json.h: a flat object of ten fixed keys, no nesting, no arrays,
+// no escapes, ~191 bytes at its widest. A full JSON parser buys nothing for
 // that shape and costs a heap-allocating JsonDocument on every GATT read — and,
 // more importantly here, it cannot be linked into a host test, which is how
 // this contract stopped being checkable in the first place.
@@ -59,6 +65,16 @@ namespace wire {
 // First protocol revision that names the expander count "mcps_healthy".
 constexpr uint8_t REV_MCPS_HEALTHY = 3;
 
+// First protocol revision carrying "idle_s" and the night-mode status bit.
+constexpr uint8_t REV_NIGHT_MODE = 4;
+
+// Status bit 0x80: the counter is deliberately not sensing. Mirrors
+// beecounter_proto::STATUS_NIGHT_IDLE in the HiveTraffic repo; duplicated
+// rather than shared because the two firmwares have no common header, and
+// pinned by test-data/test_bee_counter_wire.cpp so the pair cannot drift apart
+// silently.
+constexpr uint8_t STATUS_NIGHT_IDLE = 0x80;
+
 // One decoded measurement document. Field names are HiveHub's, normalized
 // across both revisions; `protocol_version` records which revision produced it.
 struct Measurement {
@@ -79,8 +95,21 @@ struct Measurement {
     uint32_t total_in         = 0;   // lifetime totals, saturating at UINT32_MAX
     uint32_t total_out        = 0;
     uint32_t glitch_count     = 0;   // "glitches", saturating
+    // Seconds of night-mode suspension still to run; 0 while counting, and 0
+    // on any counter older than fw:4. Non-zero means the zeros in total_in /
+    // total_out for this interval are deliberate: HiveHub asked the counter to
+    // stop, because honey bees do not fly at night. Without it that is
+    // indistinguishable from failed emitters.
+    uint32_t idle_s           = 0;
 };
 
+// Is this counter currently suspended? Reads the countdown rather than the
+// status bit, because the countdown is the value the firmware derives at
+// serialization time and therefore the one that cannot lag the other.
+inline bool isNightIdle(const Measurement& m) {
+    return m.idle_s > 0;
+}
+
 namespace detail {
 
 inline bool isSpace(char c) {
@@ -263,6 +292,8 @@ inline bool parseMeasurement(const char* json, size_t len, Measurement& out) {
             out.total_out = value;
         } else if (klen == 8 && memcmp(key, "glitches", 8) == 0) {
             out.glitch_count = value;
+        } else if (klen == 6 && memcmp(key, "idle_s", 6) == 0) {
+            out.idle_s = value;
         }
         // else: unknown key, already skipped.
     }

firmware/include/config.h

@@ -620,6 +620,20 @@
 #ifndef BEECOUNTER_GATT_CHAR_UUID
 #define BEECOUNTER_GATT_CHAR_UUID    "8e8b0102-7a1c-4b9e-9a2f-1d6e0b9c1a01"
 #endif
+// Night-mode control characteristic (HiveTraffic protocol v4 and later). We
+// write "stop sensing for N seconds" here once per cycle for as long as the
+// configured night window lasts; see night_mode.h for the decision and
+// HiveTraffic's docs/ble-mode.md for the frames. A counter too old to have it
+// simply has no such characteristic, and the write is skipped — night mode
+// degrades to "this counter keeps counting", never to an error.
+#ifndef BEECOUNTER_GATT_CONTROL_UUID
+#define BEECOUNTER_GATT_CONTROL_UUID "8e8b0103-7a1c-4b9e-9a2f-1d6e0b9c1a01"
+#endif
+// Control opcodes, mirroring beecounter_proto in the HiveTraffic repo.
+#ifndef BEECOUNTER_CTRL_OP_SET_IDLE
+#define BEECOUNTER_CTRL_OP_SET_IDLE 0x01
+#endif
+
 // OTA characteristics. Unlike HiveInside these live in the SAME service as the
 // measurement characteristic above — HiveTraffic has only one service.
 // Must match HiveTraffic Firmware/src/ble_link.cpp.

firmware/include/device_prefs.h

@@ -12,6 +12,9 @@ String wifiPassKey(int index);
 void seedPrefsFromSecretsIfNeeded();
 void loadConfigFromPrefs();
 void saveScaleConfig();
+// Persist the HiveTraffic night-mode window from the last /config fetch, so an
+// offline boot still honours it. Called from fetchRemoteConfig().
+void saveNightModePrefs();
 // Latch "the server has this device's claim" so the claim code stops riding
 // along on every upload.
 void markClaimRegistered();