Flock_Detect: GPS-tag BLE detections (v9.16) and fix payload not showing under Reconnaissance - #351
Flock_Detect: GPS-tag BLE detections (v9.16) and fix payload not showing under Reconnaissance#351stratfan wants to merge 7 commits into
Conversation
…sibility Add GPS tagging to the Flock You BLE scanner and fix the payload not appearing under Reconnaissance on the Pager. payload.sh (v9.15 -> v9.16): - Read a live fix from gpsd (gpspipe -w + jq, same pattern as gps-checker) once per scan cycle and append `lat,lon` to each detection line. - Fall back to `NO_GPS` when there is no 2D/3D fix, so the scanner still runs without a receiver or satellite lock. - Write a companion `flock_gps_<ts>.csv` (time,mac,name,lat,lon) alongside the human-readable log for map-ready output. Visibility fix: - The Pager's payload scanner treats a directory containing a subdirectory as a category, not a payload, which hid Flock_Detect from the list. - Move the ESP32 lab simulator out of the payload folder to docs/flock-lab-sim/ (outside the deployable library/ tree) so Flock_Detect is a flat leaf and renders correctly. - Document the flat-folder requirement in the payload README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v9.16 matched only the BLE device name, but most real Flock adverts carry no name (they broadcast a MAC + manufacturer data), so name-only detection missed them - parking next to a camera produced no alert. v9.17 detects on THREE signals and alerts if any fire: - MANUF: XUNTONG manufacturer ID 0x09C8 in BLE advertising data, parsed from `hcidump --raw` (byte signature FF C8 09) - the strongest tell. - OUI: advertiser MAC prefix matched against oui_list.txt (Lite-On chipset + verified Flock/Falcon/Battery prefixes) - this file was previously bundled but unused. - NAME: BLE name substring (FS Ext Battery/Penguin/Pigvision/Flock), the v9.16 behavior, kept as a third signal. Each cycle runs `hcidump --raw` and `hcitool lescan` in parallel, then merges hits by MAC (combining tags when multiple signals fire), colors by strongest signal, GPS-tags, and buzzes/flashes. Detections iterate in the main shell via a seen-file, fixing the v9.16 cross-cycle dedup gap. CSV columns are now time,mac,name_or_label,signals,lat,lon. Verified on-device: 45s live run (289 adv reports, both output files, no crash) plus an end-to-end test of the deployed detect_hits against synthetic XUNTONG/Lite-On/verified-Flock/named packets - all detected, merged, colored, and CSV-formatted correctly; non-Flock excluded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Added a second commit (v9.17) to this PR: multi-signal BLE detection. Field-testing v9.16 surfaced a real gap — it matched only the BLE device name, but most Flock adverts carry no name (MAC + manufacturer data only), so parking next to a camera produced no alert. v9.17 now detects on three signals and alerts if any fire:
Each cycle runs |
Field-testing v9.17 on drive-bys produced no detections. Two causes: BLE dwell time at driving speed (~3s) is short versus the per-cycle adapter reset + 3s inter-cycle sleep (~67% duty cycle), and it was unknown whether the cameras emit any detectable BLE at all. v9.18 addresses both: - Near-continuous scanning: adapter is brought up ONCE at startup (not reset every cycle) and the inter-cycle sleep is removed, raising the duty cycle to ~90%. Scan windows shortened to 8s so a brief fly-by is more likely to land in an active scan. Adapter self-resets after two empty cycles. - All-advert diagnostic log (flock_alladv_<ts>.csv: time,mac,manuf_id, rssi,name) records EVERY advert, not just Flock matches. Driving past a known camera now reveals exactly what it broadcasts - the definitive way to distinguish a detection gap from a camera that emits no usable BLE. Detections also carry RSSI now. Detection is unchanged (three signals: XUNTONG manuf id 0x09C8, OUI, and name) but folded into one AD-TLV-walking awk pass that decodes company id and signed RSSI without gawk-only strtonum. Verified on-device: 40s live run captured 26 devices with correct manufacturer ids (Apple/Samsung/ Schlage) and RSSI; deployed parser detects synthetic XUNTONG/OUI/name packets (combined MANUF+NAME tag included) and excludes non-Flock. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
….19) A real drive-by validated detection: a Falcon was caught by its OUI, with zero XUNTONG 0x09C8 across 1,477 devices. On this hardware the OUI list, not the manufacturer ID, is what finds Falcons. The same drive exposed three defects, all fixed and measured on-device: - I/O error flood (786 in one run). Killing `lescan` leaves LE scanning enabled in the controller, so the next cycle could not set scan parameters. Scanning is now explicitly disabled between cycles with HCI LE Set Scan Enable=0 - fast, so the high duty cycle is kept. 786 -> 0. - Alerts lagging ~30 minutes behind the sighting. Per-device dedup grepped a growing seen-file (O(n^2)): measured 79s PER CYCLE at 1,477 devices. Bulk dedup is now a single awk pass (~0ms), and Flock hits are processed BEFORE the bulk diagnostic write so buzz/LED fire immediately. - Stale detection timestamps: computed once per cycle, now taken at the moment of detection. Two platform traps found while fixing this, both documented in the README: - The Pager UI runs payloads from a COPY at /tmp/payload-<id>.sh, so pkill on the payload path never matches a UI-launched instance. A forgotten instance holds the BT adapter and invalidates any test. - No EXIT/TERM trap: background subshells (`hcidump ... &`) inherit an EXIT trap and would delete the live dedup state every cycle, and a TERM trap makes the payload SURVIVE being killed (bash resumes after a trapped signal), leaving immortal instances fighting over the adapter. Temp files are per-instance ($$) and cleaned at startup instead. Validated on-device with the adapter idle: I/O errors 0, awk errors 0, 27 devices logged, 0 malformed MACs, 0 duplicate MACs (dedup holds across cycles), and SIGTERM now kills the payload cleanly with 0 survivors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Added v9.19 — field validation plus three fixes from a real drive-by. Detection confirmed working: a Falcon was caught by its OUI, with zero XUNTONG The same drive exposed three defects, now fixed and measured on-device:
Two platform traps worth flagging for other payload authors (both documented in the README):
Validated with the adapter idle: 0 I/O errors, 0 awk errors, 27 devices logged, 0 malformed MACs, 0 duplicate MACs across cycles, and SIGTERM kills cleanly with 0 survivors. |
There was a problem hiding this comment.
These files are being moved to the wrong directly, all files must withing the payload folder library/user/reconnaissance/Flock_Detect/*
Addresses review on hak5#351: all payload files must live within library/user/reconnaissance/Flock_Detect/. Moves the companion lab simulator out of docs/flock-lab-sim/ into the payload folder as FLAT files (flock_lab_sim.py + LAB_SIMULATOR.md, renamed from README.md to avoid colliding with the payload README). Kept flat, not in a flock-lab-sim/ subfolder, because the Pager's payload scanner treats any directory containing a subdirectory as a category and hides the payload — the original reason the sim lived under docs/. The .py runs on ESP32/MicroPython, not the Pager, and is inert there. README file list and the flat-folder note updated accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pothesis The "a Falcon was caught by its OUI / the OUI list is what finds Falcons" claim has not held up: two drive-bys, one stationary ~6-min capture within ~30 m of a confirmed camera on a solid GPS fix, and several weeks of routine driving all found no Flock advertising on any channel. Scanner was working (consumer APs to -79 dBm, 800+ BLE devices at the site), so the cameras most likely emitted nothing. Reframe the claim as not-yet-reproduced and add the working hypothesis that operating Falcons are cellular-only (Sierra LTE), with WiFi/BLE used only at install — making the FLOCK_VERIFIED OUIs likely mid-install captures and deflock.me the more reliable locator. Also correct the "primarily a WiFi device" line. No code change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @hak5peaks — moved both lab-sim files into the payload folder ( One layout note: I kept them as flat files rather than a Separately, a field-results update while I'm here: I've been driving with this for several weeks and have not seen a Flock camera advertising on any channel in that time — no WiFi OUI match, no XUNTONG |
…lder Same convention as the lab simulator (per the hak5#351 review): payload docs live inside library/user/reconnaissance/Flock_Detect/ as flat files, not under a top-level docs/ tree. Moves docs/flock-hits/ground-truth-capture.md to Flock_Detect/GROUND_TRUTH_CAPTURE.md and updates the two README links to the in-folder path. (docs/flock-hits/flock_hits.sh is a separate concern and stays put.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Summary
Two changes to the Flock_Detect ("Flock You") reconnaissance payload:
gpsd, and a machine-readable CSV is written alongside the existing log.Visibility fix — root cause
The Pager's payload scanner treats any directory that contains a subdirectory as a category (to recurse into) rather than a payload, and lists a folder as a payload only when it's a flat leaf containing
payload.sh.Upstream,
Flock_Detect/contains aflock_lab_sim/subfolder (the ESP32 companion simulator), so the scanner classifiesFlock_Detectas a category, ignores its ownpayload.sh, and — sinceflock_lab_sim/has nopayload.sh— nothing is listed. The payload is effectively invisible in the UI.Fix: move the ESP32 lab simulator out of the deployable payload folder to
docs/flock-lab-sim/(outside thelibrary/tree).Flock_Detect/becomes a flat leaf and renders correctly. Verified on-device: after flattening, "Flock You" appears under Reconnaissance. The flat-folder requirement is now documented in the payload README.The simulator runs on separate Arduino/ESP32 hardware, not the Pager, so it does not belong in the deployable payload directory.
GPS tagging details (v9.16)
gpspipe -w+jq(the same pattern used bygps-checker) — position doesn't change meaningfully within a ~12s scan window, and this avoids multiplying GPS delay across multiple detections.lat,lonto each on-screen detection line and to the log file.NO_GPSwhen there's no 2D/3D fix (no receiver / no lock yet), so the scanner still runs normally without GPS.flock_gps_<timestamp>.csv(time,mac,name,lat,lon) for map-ready output.Testing
bash -nclean.get_gps_fix()exercised against a livegpsdon a WiFi Pineapple Pager — correctly returnedNO_GPSatmode:1(no lock) and correctlat,lonon a valid fix..txtand.csvoutputs, no crash.🤖 Generated with Claude Code