Firefly-inspired visual synchronisation for decentralised ground-robot coordination

MSc Project — BIOE70025

This repository studies whether agents can coordinate their timing from local visual LED flashes and camera detection, without a central timing controller. The platform evolved from early drone-oriented planning to a safer physical ground-robot/HIL testbed; the research question did not change.

Start here: final thesis package

The authoritative final-thesis package is step8_thesis_mainline. It is the evidence map for supervisory review and the intended basis of the final GitHub package; historical worktrees are not required to inspect the final narrative.

Thesis question	Canonical thesis location
Model selection: EAPF, Kuramoto, and biological PRC/LCO comparison	step8_thesis_mainline/8a_model_selection/
Physical N=2 synchronisation and physical N=3 all-to-all/directed/bidirectional topologies	step8_thesis_mainline/8b_synchronisation/
Final eight-condition, 80-trial robustness campaign	step8_thesis_mainline/8c_robustness/
Centralised versus decentralised action timing	step8_thesis_mainline/8d_baseline_comparison/
Thesis source, figures, bibliography, and figure-generation code	step8_thesis_mainline/thesis_writing/
Cross-area analysis utilities and claim-to-evidence manifest	step8_thesis_mainline/analysis/ and step8_thesis_mainline/THESIS_CANONICAL_MANIFEST.md

The principal controller conclusion is deliberately scoped: EAPF is better suited than Kuramoto to this project's binary-flash, camera-mediated, event-based HIL setting. It is not a claim of universal superiority.

Canonical Step 8D video policy

The only canonical raw videos for the baseline/action-timing result are the ten centralised videos in 8d_baseline_comparison/centralized_original/ and the ten decentralised videos in 8d_baseline_comparison/decentralized_original/. The former 240-fps topology campaign remains only as SUPERSEDED / NON-CANONICAL VIDEO ANALYSIS PROVENANCE; its video containers are not thesis evidence and were removed under the recorded 2026-08-31 policy.

GitHub and raw-data boundary

Source code, tests, configurations, analysis scripts, compact result tables, figures, thesis source, and manifests are intended to be readable in Git. Large raw experimental payloads are intended for a separately deposited archive with SHA-256 verification rather than ordinary Git blobs. See step8_thesis_mainline/GITHUB_PACKAGING_MANIFEST.csv and step8_thesis_mainline/GITHUB_DATA_ARCHIVE_RESTORATION.md for the precise staging plan and restoration requirements.

The pinned led_firesync biological-comparator snapshot has no local LICENSE/COPYING/NOTICE evidence. Its upstream source is therefore retained locally for provenance but must not be published until redistribution permission is established; the public package retains only its provenance, commit, citation, and SHA-256 record.

Known limitation

The retained scalability material is marked REQUIRES_RERUN. It can be reviewed as provenance and research history, but it must not be presented as a clean final reproducible thesis result until compatible simulator-engine provenance is established.

Repository source layout

firefly-sync/
├── firefly_sync/              # Project-authored models and hardware interfaces
├── experiments/               # Project-authored runners and analyses
├── tests/                     # Unit and integration tests
└── step8_thesis_mainline/     # Canonical final-thesis package

Historical development and hardware reference

The following operational material documents earlier HIL development. It is not a GitHub submission runbook and should not be used to launch hardware as part of package review.

Raspberry Pi 5 Visual Flash Detection

This section covers the hardware-in-the-loop testbed: a laptop browser displays a flashing leader target, a Raspberry Pi 5 camera observes it, and the Pi detects flash events in real time.

Hardware Setup

• Raspberry Pi 5 with Arducam 8MP (imx219) on CAM/DISP 0
• GPIO17 → physical LED through a current-limiting resistor
• Laptop screen displaying the leader UI
• Pi camera positioned to observe the laptop screen at short range

1. Copy Project to Pi

# From the laptop:
scp -r firefly-sync pi@dronepi.local:~/

2. Install Pi Dependencies (on the Pi)

Preferred — apt packages (Raspberry Pi OS):

sudo apt install -y python3-flask python3-opencv python3-picamera2 python3-gpiozero

Alternative — pip:

pip install flask opencv-python picamera2 gpiozero

3. Test the LED (on the Pi)

cd ~/firefly-sync
python experiments/test_pi_led.py --pin 17 --cycles 5

The physical LED on GPIO17 should blink 5 times.

4. Run Leader UI (on the laptop)

cd firefly-sync
python experiments/run_leader_ui.py

Or open experiments/leader_ui/index.html directly in a browser.

5. Start Pi Camera Detection Server (on the Pi)

Recommended — top-percentile mode with auto-ROI:

cd ~/firefly-sync
PYTHONPATH=. python3 experiments/stream_pi_camera_detection.py \
    --host 0.0.0.0 --port 5000 \
    --detection-mode top_percentile --percentile 99 \
    --threshold-on 180 --threshold-off 120 --min-interval 0.2 \
    --auto-roi --auto-roi-duration 3

Manual ROI (if you know the target's pixel position):

PYTHONPATH=. python3 experiments/stream_pi_camera_detection.py \
    --host 0.0.0.0 --port 5000 \
    --detection-mode top_percentile \
    --roi 200 120 240 240 --percentile 99 \
    --threshold-on 180 --threshold-off 120 --min-interval 0.2

Detection modes:

• top_percentile — robust for small targets on dark backgrounds (default)
• mean — full-frame or ROI mean brightness
• bright_blob — largest bright connected component

6. Connect Leader UI to Pi Stream

1. In the leader UI, locate the Pi Camera Monitor panel (right side).
2. Ensure the URL field shows http://dronepi.local:5000 (edit as needed, e.g. http://192.168.1.127:5000).
3. Press Connect.
4. The MJPEG video feed should appear with detection overlay.
5. Press ◎ Auto Locate Flash Region to automatically find the flashing target (keep the leader flashing and the camera still for 3 seconds).
6. Status readouts update in real time.

Expected Signs of Successful Detection

• Camera stream visible in the right panel with ROI rectangle
• ROI source shows auto or manual after auto-localisation
• Bright (used) changes from dark to bright when the leader flashes ON
• Top %ile brightness is high during ON phases even with a small target
• State switches between OFF and ON, synchronized with the leader flash
• Rising edge count increases by 1 per ON transition
• Estimated frequency approaches the leader UI frequency (e.g. ~1 Hz)
• CSV log saved under experiments/logs/pi_camera_stream_YYYYMMDD_HHMMSS.csv

CSV Log Columns

timestamp_s, elapsed_time_s, frame_index, detection_mode,
brightness_used, full_frame_mean, top_percentile_brightness,
brightness_mean, state, event_type, rising_edge_count,
estimated_frequency_hz, roi, roi_source, roi_confidence,
percentile, blob_found, blob_area_px, blob_bbox,
threshold_on, threshold_off

One row is written per processed frame. The event_type column is empty for normal frames and "leader_rising_edge" when a valid rising edge is detected.

Troubleshooting: Rising Edge Count Does Not Increase

If the camera stream is visible but rising_edge_count stays at 0:

1. Fix the camera physically — point it directly at the laptop screen.
2. Set leader UI to high contrast — ON brightness 255, OFF brightness 0, background brightness 0 (the default).
3. Use a large target size initially — 150–250 px square, centred.
4. Start leader flashing at 1 Hz with 50% duty cycle.
5. Run the Pi server with local_contrast + adaptive mode (recommended):

   PYTHONPATH=. python3 experiments/stream_pi_camera_detection.py \
       --host 0.0.0.0 --port 5000 \
       --detection-mode local_contrast --auto-roi --auto-roi-duration 3 \
       --window-s 5 --norm-on-threshold 0.65 --norm-off-threshold 0.35 \
       --min-interval 0.2

6. Optional — fix camera exposure to prevent auto-exposure from fighting the flashing signal:

   PYTHONPATH=. python3 experiments/stream_pi_camera_detection.py \
       --host 0.0.0.0 --port 5000 \
       --detection-mode local_contrast --auto-roi --auto-roi-duration 3 \
       --manual-camera --exposure-us 8000 --analogue-gain 1.0 \
       --awb-enable false --target-fps 30

7. Click "Auto Locate Flash Region" in the leader UI camera panel.
8. Check the signal_norm plot in the camera panel:
   • If the trace is a clear square-ish wave swinging 0→1, detection is working.
   • If the trace is flat → ROI/camera positioning is wrong, or the target is not flashing with enough contrast, or camera auto-exposure is cancelling out the flash.
   • If the trace is noisy → ambient light or screen reflections are interfering; increase target size or move camera closer.
9. Check adaptive thresholds in the overlay/readouts:
   • adaptive_low should be near the dark-phase local_contrast value.
   • adaptive_high should be near the bright-phase local_contrast value.
   • adaptive_amplitude should be comfortably above min_amplitude (default 10).
10. If state is stuck ON, lower --norm-off-threshold (e.g. 0.25) or check that the signal drops below adaptive_low during the OFF phase.
11. If state is stuck OFF, raise --norm-on-threshold (e.g. 0.55) or check that adaptive_amplitude exceeds --min-amplitude.

Interpreting the Signal Norm Plot

The signal_norm plot in the leader UI camera panel shows the last ~10 seconds of normalised detection signal:

• Green dashed line at ~0.65 — ON threshold. When the trace crosses above this line from below, a rising edge is detected.
• Red dashed line at ~0.35 — OFF threshold. When the trace drops below this line from above, the state switches to OFF.
• Cyan trace — the normalised signal (0 = dark, 1 = bright).

A working system shows a square-ish waveform swinging between ~0 and ~1, crossing both thresholds once per flash cycle.

Tuning Adaptive Thresholds

Symptom	Likely Cause	Fix
State stuck OFF, trace swings 0→1	norm_on_threshold too high	Lower --norm-on-threshold to 0.55
State stuck ON, trace swings 1→0	norm_off_threshold too low	Raise --norm-off-threshold to 0.45
Trace flat near 0	ROI wrong / no flash in frame / camera dark	Reposition camera, enlarge target, check leader UI is flashing
Trace flat near 0.5	Camera auto-exposure cancelling flash	Use --manual-camera with fixed exposure
Trace periodic but amplitude small	Target too small or dim	Enlarge target, increase ON brightness
signal_quality stays 0	min_amplitude too high	Lower --min-amplitude or check signal_amplitude readout
periodicity_confidence low	Signal noisy or irregular	Improve lighting, reduce screen reflections
signal_frequency_hz near 0 but edges fire	Not enough autocorrelation data	Wait 5+ seconds for history to build

Known Limitations

• Laptop and Pi clocks are not synchronised — compare detection timestamps with Pi-side perf_counter, not browser performance.now().
• The browser MJPEG stream is for debugging only; future synchronisation must use Pi-side detection timestamps.
• MJPEG streaming introduces ~100–200 ms latency; detection is real-time on the Pi but display on the laptop lags slightly.
• Fullscreen mode hides both the control panel and the camera panel. Use windowed mode to monitor detection while the leader runs.

Authorship

Built as part of the MSc in Bioengineering at Imperial College London.