Skip to content

Flock_Detect: GPS-tag BLE detections (v9.16) and fix payload not showing under Reconnaissance - #351

Open
stratfan wants to merge 7 commits into
hak5:masterfrom
stratfan:flock-detect-gps-v9.16
Open

Flock_Detect: GPS-tag BLE detections (v9.16) and fix payload not showing under Reconnaissance#351
stratfan wants to merge 7 commits into
hak5:masterfrom
stratfan:flock-detect-gps-v9.16

Conversation

@stratfan

Copy link
Copy Markdown
Contributor

Summary

Two changes to the Flock_Detect ("Flock You") reconnaissance payload:

  1. GPS tagging (v9.15 → v9.16). Each BLE detection is now tagged with a live GPS fix from gpsd, and a machine-readable CSV is written alongside the existing log.
  2. Visibility fix. Flock_Detect currently does not appear under Reconnaissance on the Pager because it contains a nested subdirectory. This flattens the payload folder so it renders.

Visibility fix — root cause

The Pager's payload scanner treats any directory that contains a subdirectory as a category (to recurse into) rather than a payload, and lists a folder as a payload only when it's a flat leaf containing payload.sh.

Upstream, Flock_Detect/ contains a flock_lab_sim/ subfolder (the ESP32 companion simulator), so the scanner classifies Flock_Detect as a category, ignores its own payload.sh, and — since flock_lab_sim/ has no payload.sh — nothing is listed. The payload is effectively invisible in the UI.

Fix: move the ESP32 lab simulator out of the deployable payload folder to docs/flock-lab-sim/ (outside the library/ tree). Flock_Detect/ becomes a flat leaf and renders correctly. Verified on-device: after flattening, "Flock You" appears under Reconnaissance. The flat-folder requirement is now documented in the payload README.

The simulator runs on separate Arduino/ESP32 hardware, not the Pager, so it does not belong in the deployable payload directory.

GPS tagging details (v9.16)

  • Reads one fix per scan cycle via gpspipe -w + jq (the same pattern used by gps-checker) — position doesn't change meaningfully within a ~12s scan window, and this avoids multiplying GPS delay across multiple detections.
  • Appends lat,lon to each on-screen detection line and to the log file.
  • Falls back to NO_GPS when there's no 2D/3D fix (no receiver / no lock yet), so the scanner still runs normally without GPS.
  • Writes a companion flock_gps_<timestamp>.csv (time,mac,name,lat,lon) for map-ready output.

Testing

  • bash -n clean.
  • Both tag branches (fix / no-fix) unit-tested.
  • get_gps_fix() exercised against a live gpsd on a WiFi Pineapple Pager — correctly returned NO_GPS at mode:1 (no lock) and correct lat,lon on a valid fix.
  • Full payload run on-device (timeout-bounded): starts cleanly, survives scan cycles, creates both .txt and .csv outputs, no crash.
  • Confirmed the flattened folder now lists under Reconnaissance on the device.

🤖 Generated with Claude Code

stratfan and others added 2 commits August 21, 2026 23:05
…sibility

Add GPS tagging to the Flock You BLE scanner and fix the payload not
appearing under Reconnaissance on the Pager.

payload.sh (v9.15 -> v9.16):
- Read a live fix from gpsd (gpspipe -w + jq, same pattern as gps-checker)
  once per scan cycle and append `lat,lon` to each detection line.
- Fall back to `NO_GPS` when there is no 2D/3D fix, so the scanner still
  runs without a receiver or satellite lock.
- Write a companion `flock_gps_<ts>.csv` (time,mac,name,lat,lon) alongside
  the human-readable log for map-ready output.

Visibility fix:
- The Pager's payload scanner treats a directory containing a subdirectory
  as a category, not a payload, which hid Flock_Detect from the list.
- Move the ESP32 lab simulator out of the payload folder to
  docs/flock-lab-sim/ (outside the deployable library/ tree) so
  Flock_Detect is a flat leaf and renders correctly.
- Document the flat-folder requirement in the payload README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v9.16 matched only the BLE device name, but most real Flock adverts carry
no name (they broadcast a MAC + manufacturer data), so name-only detection
missed them - parking next to a camera produced no alert.

v9.17 detects on THREE signals and alerts if any fire:
- MANUF: XUNTONG manufacturer ID 0x09C8 in BLE advertising data, parsed
  from `hcidump --raw` (byte signature FF C8 09) - the strongest tell.
- OUI: advertiser MAC prefix matched against oui_list.txt (Lite-On chipset
  + verified Flock/Falcon/Battery prefixes) - this file was previously
  bundled but unused.
- NAME: BLE name substring (FS Ext Battery/Penguin/Pigvision/Flock), the
  v9.16 behavior, kept as a third signal.

Each cycle runs `hcidump --raw` and `hcitool lescan` in parallel, then
merges hits by MAC (combining tags when multiple signals fire), colors by
strongest signal, GPS-tags, and buzzes/flashes. Detections iterate in the
main shell via a seen-file, fixing the v9.16 cross-cycle dedup gap. CSV
columns are now time,mac,name_or_label,signals,lat,lon.

Verified on-device: 45s live run (289 adv reports, both output files, no
crash) plus an end-to-end test of the deployed detect_hits against
synthetic XUNTONG/Lite-On/verified-Flock/named packets - all detected,
merged, colored, and CSV-formatted correctly; non-Flock excluded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@stratfan

Copy link
Copy Markdown
Contributor Author

Added a second commit (v9.17) to this PR: multi-signal BLE detection.

Field-testing v9.16 surfaced a real gap — it matched only the BLE device name, but most Flock adverts carry no name (MAC + manufacturer data only), so parking next to a camera produced no alert. v9.17 now detects on three signals and alerts if any fire:

  • MANUF — XUNTONG manufacturer ID 0x09C8 in the advertising data (hcidump --raw, signature FF C8 09) — the strongest tell, and the one that catches no-name devices.
  • OUI — advertiser MAC prefix matched against the bundled oui_list.txt (Lite-On chipset + verified Flock prefixes), which was previously unused.
  • NAME — the original v9.16 substring match, kept as a third signal.

Each cycle runs hcidump --raw + hcitool lescan in parallel and merges hits by MAC. Verified on-device (45s live run, 289 adv reports, no crash) plus an end-to-end test of the detection function against synthetic XUNTONG/Lite-On/verified-Flock/named packets.

stratfan and others added 2 commits August 26, 2026 16:52
Field-testing v9.17 on drive-bys produced no detections. Two causes:
BLE dwell time at driving speed (~3s) is short versus the per-cycle
adapter reset + 3s inter-cycle sleep (~67% duty cycle), and it was
unknown whether the cameras emit any detectable BLE at all.

v9.18 addresses both:
- Near-continuous scanning: adapter is brought up ONCE at startup (not
  reset every cycle) and the inter-cycle sleep is removed, raising the
  duty cycle to ~90%. Scan windows shortened to 8s so a brief fly-by is
  more likely to land in an active scan. Adapter self-resets after two
  empty cycles.
- All-advert diagnostic log (flock_alladv_<ts>.csv: time,mac,manuf_id,
  rssi,name) records EVERY advert, not just Flock matches. Driving past a
  known camera now reveals exactly what it broadcasts - the definitive
  way to distinguish a detection gap from a camera that emits no usable
  BLE. Detections also carry RSSI now.

Detection is unchanged (three signals: XUNTONG manuf id 0x09C8, OUI, and
name) but folded into one AD-TLV-walking awk pass that decodes company id
and signed RSSI without gawk-only strtonum. Verified on-device: 40s live
run captured 26 devices with correct manufacturer ids (Apple/Samsung/
Schlage) and RSSI; deployed parser detects synthetic XUNTONG/OUI/name
packets (combined MANUF+NAME tag included) and excludes non-Flock.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
….19)

A real drive-by validated detection: a Falcon was caught by its OUI, with
zero XUNTONG 0x09C8 across 1,477 devices. On this hardware the OUI list,
not the manufacturer ID, is what finds Falcons.

The same drive exposed three defects, all fixed and measured on-device:

- I/O error flood (786 in one run). Killing `lescan` leaves LE scanning
  enabled in the controller, so the next cycle could not set scan
  parameters. Scanning is now explicitly disabled between cycles with HCI
  LE Set Scan Enable=0 - fast, so the high duty cycle is kept. 786 -> 0.
- Alerts lagging ~30 minutes behind the sighting. Per-device dedup grepped
  a growing seen-file (O(n^2)): measured 79s PER CYCLE at 1,477 devices.
  Bulk dedup is now a single awk pass (~0ms), and Flock hits are processed
  BEFORE the bulk diagnostic write so buzz/LED fire immediately.
- Stale detection timestamps: computed once per cycle, now taken at the
  moment of detection.

Two platform traps found while fixing this, both documented in the README:
- The Pager UI runs payloads from a COPY at /tmp/payload-<id>.sh, so
  pkill on the payload path never matches a UI-launched instance. A
  forgotten instance holds the BT adapter and invalidates any test.
- No EXIT/TERM trap: background subshells (`hcidump ... &`) inherit an
  EXIT trap and would delete the live dedup state every cycle, and a TERM
  trap makes the payload SURVIVE being killed (bash resumes after a
  trapped signal), leaving immortal instances fighting over the adapter.
  Temp files are per-instance ($$) and cleaned at startup instead.

Validated on-device with the adapter idle: I/O errors 0, awk errors 0,
27 devices logged, 0 malformed MACs, 0 duplicate MACs (dedup holds across
cycles), and SIGTERM now kills the payload cleanly with 0 survivors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@stratfan

Copy link
Copy Markdown
Contributor Author

Added v9.19 — field validation plus three fixes from a real drive-by.

Detection confirmed working: a Falcon was caught by its OUI, with zero XUNTONG 0x09C8 across 1,477 devices logged that drive. On this hardware the OUI list, not the manufacturer ID, is what finds Falcons.

The same drive exposed three defects, now fixed and measured on-device:

Symptom Cause Result
Set scan parameters failed: I/O error ×786 killing lescan leaves LE scanning enabled, so the next cycle can't set scan params explicit LE Set Scan Enable=0 between cycles → 786 → 0
alert buzzed ~30 min after the sighting per-device dedup grepped a growing seen-file (O(n²)) — measured 79 s per cycle at 1,477 devices single awk pass (~0 ms); hits processed before the bulk write
timestamps stale by minutes computed once per cycle taken at the moment of detection

Two platform traps worth flagging for other payload authors (both documented in the README):

  • The Pager UI runs a payload from a copy at /tmp/payload-<id>.sh, so pkill -f '<name>/payload.sh' never matches a UI-launched instance — a forgotten one holds the BT adapter and silently invalidates testing.
  • This payload deliberately has no EXIT/TERM trap: background subshells inherit an EXIT trap (which would delete live dedup state each cycle), and a TERM trap makes the payload survive being killed, leaving immortal instances contending for the adapter.

Validated with the adapter idle: 0 I/O errors, 0 awk errors, 27 devices logged, 0 malformed MACs, 0 duplicate MACs across cycles, and SIGTERM kills cleanly with 0 survivors.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These files are being moved to the wrong directly, all files must withing the payload folder library/user/reconnaissance/Flock_Detect/*

@hak5peaks hak5peaks added the Pending requested changes Awaiting changes from author label Aug 29, 2026
stratfan and others added 2 commits September 6, 2026 14:18
Addresses review on hak5#351: all payload files must live within
library/user/reconnaissance/Flock_Detect/. Moves the companion lab
simulator out of docs/flock-lab-sim/ into the payload folder as FLAT
files (flock_lab_sim.py + LAB_SIMULATOR.md, renamed from README.md to
avoid colliding with the payload README).

Kept flat, not in a flock-lab-sim/ subfolder, because the Pager's payload
scanner treats any directory containing a subdirectory as a category and
hides the payload — the original reason the sim lived under docs/. The .py
runs on ESP32/MicroPython, not the Pager, and is inert there. README file
list and the flat-folder note updated accordingly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pothesis

The "a Falcon was caught by its OUI / the OUI list is what finds Falcons"
claim has not held up: two drive-bys, one stationary ~6-min capture within
~30 m of a confirmed camera on a solid GPS fix, and several weeks of routine
driving all found no Flock advertising on any channel. Scanner was working
(consumer APs to -79 dBm, 800+ BLE devices at the site), so the cameras most
likely emitted nothing.

Reframe the claim as not-yet-reproduced and add the working hypothesis that
operating Falcons are cellular-only (Sierra LTE), with WiFi/BLE used only at
install — making the FLOCK_VERIFIED OUIs likely mid-install captures and
deflock.me the more reliable locator. Also correct the "primarily a WiFi
device" line. No code change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@stratfan

stratfan commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @hak5peaks — moved both lab-sim files into the payload folder (ba3e809). They now sit flat alongside the payload:

library/user/reconnaissance/Flock_Detect/
  payload.sh
  oui_list.txt
  README.md
  flock_lab_sim.py     # ESP32 lab simulator (runs on Arduino, not the Pager)
  LAB_SIMULATOR.md     # its setup guide (renamed from README.md to avoid a collision)

One layout note: I kept them as flat files rather than a flock-lab-sim/ subfolder on purpose — the Pager's payload scanner treats any payload directory that contains a subdirectory as a category and hides the payload from the on-device list (that was the original reason these lived under docs/). Flat files inside Flock_Detect/ avoid that while keeping everything in the payload folder as you asked. The .py runs on ESP32/MicroPython and is inert on the Pager.

Separately, a field-results update while I'm here: I've been driving with this for several weeks and have not seen a Flock camera advertising on any channel in that time — no WiFi OUI match, no XUNTONG 0x09C8, no Flock-/Falcon-style SSID, and no BLE hits — including a stationary capture parked right next to a confirmed camera. Two possibilities: the deployed Falcons are cellular-only (Sierra LTE) and only beacon WiFi/BLE during install; and/or — with the recent public scrutiny over privacy and unauthorized use — WiFi/BLE may have been deliberately disabled on deployed units to reduce their RF detectability. Either way the practical result is the same, so I've toned down the "detection confirmed" language in the README so the OUI list's confidence isn't overstated. Flagging in case others are seeing the same.

…lder

Same convention as the lab simulator (per the hak5#351 review): payload docs
live inside library/user/reconnaissance/Flock_Detect/ as flat files, not
under a top-level docs/ tree. Moves docs/flock-hits/ground-truth-capture.md
to Flock_Detect/GROUND_TRUTH_CAPTURE.md and updates the two README links to
the in-folder path. (docs/flock-hits/flock_hits.sh is a separate concern and
stays put.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Pending requested changes Awaiting changes from author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants