RSH Vibroarthrography App

This project was created as part of the SEKMO program in collaboration with the Centre for Environmental Intelligence and Sensing at Taltech

Desktop research application for capturing, visualising, analysing and sonifying vibroarthrography (VAG) signals from the Nexus N3 HDR Dot sensor.

The application supports live BLE streaming, real-time scalogram visualisation and audio monitoring, together with capture export, offline signal analysis and audified playback of saved VAG recordings.

The current entry point is app.py. The application uses customtkinter for the user interface, matplotlib for live and offline visualisation, and the local vag_sensor_manager package for BLE device discovery and control.

Current Scope

• Discover and connect to a single Nexus N3 HDR Dot BLE sensor.

• Stream acceleration-derived VAG signals at 2048 Hz.

• Display a rolling live scalogram during streaming.

• Listen to the live VAG signal through real-time audification.

• Capture fixed-length signal segments and save them to CSV.

• Analyse saved captures using:

  • raw signal visualisation
  • scalogram visualisation
  • frequency-band power analysis
  • fractal-dimension estimation

• Sonify saved captures using selectable direct, speed-adjusted or heterodyne playback.

Audio Sonification

The application converts VAG vibration signals into audible sound, allowing temporal and frequency characteristics to be explored by listening as well as through visual analysis.

Audio is available in two parts of the application:

• Live audio monitoring: enable Audio before starting a BLE stream to listen to the incoming VAG signal in real time.
• Saved-capture playback: open a CSV capture in the Analyse view and use the playback controls to listen to the recorded signal.

Three playback methods are available:

• Direct: preserves the original timing while resampling the signal for audio output.
• Speed: accelerates the signal, shifting lower-frequency vibration content upward and reducing playback duration.
• Heterodyne: frequency-shifts the signal into a more readily audible range while retaining its temporal structure.

Sonification is provided as an exploratory research and signal-inspection tool. It is not intended to provide a clinical or diagnostic interpretation.

Repository Layout

• app.py: main GUI entry point
• components/stream/stream.py: streaming UI, capture flow, live visualization
• components/analyse/analyse.py: offline analysis and playback
• settings/settings.py: application defaults and export paths
• modules/processing_pipeline.py: filtering and analysis helpers
• modules/audio_processor.py: real-time audio processing
• modules/data_streamer.py: capture/export helpers
• vag_sensor_manager/: BLE device manager, adapters, packets, and device abstractions

Requirements

Python 3.10+ is recommended.

Install Python dependencies:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The current requirements.txt is minimal and does not pin versions. On Linux BLE setups you may also need system Bluetooth and DBus support available to Python because the app depends on bleak and dbus-python.

Running The App

From the project root:

python3 app.py

Default runtime behavior:

• Window size is fixed to 1280x800.
• Dark customtkinter theme is enabled.
• BLE is the default and only active device path in the current UI.
• Export directories are created automatically under ~/Desktop/VAG_Exports.

Generated output folders:

• ~/Desktop/VAG_Exports/captures
• ~/Desktop/VAG_Exports/recordings

Typical Workflow

1. Launch the app.
2. Open Find Device.
3. Select the discovered Nexus N3 HDR Dot device from the dropdown.
4. Start streaming.
5. Optionally capture a segment for later analysis.
6. Switch to Analyse to inspect a saved capture and play back the audified signal.

Settings Exposed In The UI

• metronome toggle
• half-cycle time for capture length
• BLE stream mode (MAG, X, Y, Z, XYZ, ALL)
• audio playback mode
• filter low cutoff
• filter high cutoff
• filter order

Building A Standalone Executable

PyInstaller spec file:

• app_build.spec

Build command:

pyinstaller --clean app_build.spec

The spec includes:

• assets/
• vag_sensor_manager/

and names the collected application rsh-vibroarthrography.

Review Notes

The review focused on the current GUI path and supporting modules rather than previously built artifacts under dist/.

Confirmed areas:

• The app compiles successfully with python3 -m compileall ., although that command also walks committed build output under dist/.
• The current code path is centered on BLE streaming, capture export, offline analysis, and audio playback.

Known issues in the current code:

• Settings still exposes several getters for fields that are never initialized in the default path, including spectrogram-related values in settings/settings.py. They are mostly shielded by commented-out UI, but the configuration layer is incomplete.

Limitations

• The UI is effectively single-device despite some lower-level manager support for multiple devices.
• USB/serial support is present as legacy scaffolding but is not the active path in the current app.
• The README describes the code as it exists now; it does not guarantee that all hardware-specific BLE flows can be exercised without a Nexus N3 HDR Dot device and a compatible Bluetooth stack.