Author: Sina Röllin / Spring 2025
- Overview
- Control
- GUI (Graphical User Interface)
- Development Mode
- Prerequisites
- Installation
- Libraries
- References
This software translates, integrates and simplifies previously separated control and tracking systems into a unified Python-based graphical user interface (GUI). The original setup included MATLAB code for magnet control and Xbox input, and Python code for visual tracking.
The software is organised into two main sections:
- Control – Handles physical actuation including catheter advancement, magnetic field orientation, and positioning of the magnetic coil.
- GUI – Provides an interactive interface for steering, tracking and visualisation.
Additionally, a Development Mode enables software simulation without the physical setup, making testing and development more accessible.
Each of the following modules runs in its own thread to ensure real-time updates and thread safety. They manage the position and orientation of the magnet as well as the movement of the catheter and the steering overall.
The Advancer class enables manual control of the catheter movement with an Xbox controller. It translates Xbox Controller inputs to serial commands sent to the Arduino and allows gradual advancing and reversing of the catheter. Its speed is calculated based on the computations by Mehdi Krichen. [2]
See: Advancer.py
The MagnetOrientation class enables real-time control of the magnetic field used to steer the magnetic catheter head. It converts Xbox controller input or the manual input from the GUI to the correct orientation commands for the magnet within the magnetic coil system. It implements the computations by Mehdi Krichen and Julian Raub. [2],[3]
See: MagnetOrientation.py
The PIController class interfaces with a Physik Instrumente (PI) controller to manipulate the 3-axis stage that positions the magnetic coil above the phantom. It uses the Xbox controller input or the manual input from the GUI to move the stage within the 100×100×100 mm³ workspace along the x, y, and z axes. There are additional safety features in place that minimise the magnetic coil from colliding with the phantom. The controller always moves the z-axis first and enforces safety constraints by limiting and clipping inputs accordingly.
See: PIController.py
The XboxController class handles input from the Xbox controller to manually steer the magnet, adjust the magnet orientation and control the advancer. It reads joystick, hat and button inputs and produces a joystick state dictionary that other modules use for real-time control.
See: XboxController.py
The GUI provides an intuitive interface for interacting with the system, allowing users to control and visualise the system. It is organised into five main groups, each responsible for steering or visualising a specific aspect of the system (implemented in Gui.py). (see Gui.py)
The five GUI groups are:
- Position
- Automation
- Advancer
- Magnet Orientation
- Visual Tracking (featured as the camera frames and toggle button)
This section of the GUI enables manual control of the PI controller’s position by allowing the user to input values for the x, y, and z coordinates directly.
This widget enables the above mentioned steering of the position and formats it for the GUI. It handles the input fields and button in the GUI to facilitate direct user interaction with the PI controller’s position.
This section of the GUI allows users to manually enter the desired phantom number, which by pressing the "Activate Automation Step" button triggers the system to automatically position and orient the magnet according to predefined values associated with that phantom. These values are retrieved from a save.txt file containing position and orientation data provided by Mehdi Krichen.
- Target Position: Sent to the PI controller.
- Magnet Orientation: Sent to the magnet orientation controller.
Additionally, this section displays the Magnetic Orientation Vector in real time:
- Desired: The target orientation set for the magnet.
- Measured: The system’s current magnetic field vector, shown for user reference and verification.
This part of the GUI displays the current state and speed of the Advancer system.
This section of the GUI provides visual displays of the magnet's current orientation and the desired magnetization vector. Since the magnet's orientation is not physically visible, this feature is crucial to interpret the system's behaviour in real time.
The display includes three 2D projections of the magnetic vectors:
- XY Plane
- XZ Plane
- YZ Plane
Each plot shows:
- The desired Magnet Orientation Vector (in red)
- The measured Magnet Orientation Vector (in blue)
This module handles the above mentioned plotting functionalities.
see: MagnetPlotWidget.py
This section of the GUI displays the live camera feeds of the system and visual tracking of the magnetic head using computer vision techniques based on the work by Gael Gerber. [4]
The AnimatedToggle class defines the toggle system used to turn the tracking on or off.
see: AnimatedToggle.py
The Tracking Manager class detects and tracks the magnetic head in real time using the frames coming from the two Basler cameras. The following file is responsible for the implementation of the tracking feature in the real-time video frame based on the mentioned work by Gael Gerber.
see: Tracking.py
The VideoStreamWidget class manages live video display from either a Basler camera or a video file depending on the state of the development mode. It supports zooming and mouse tracking for interactive viewing as well as automatic lighting adjustment.
see: VideoStream.py
The development mode enables simulation of the GUI and visual tracking functionalities without requiring the physical system or hardware to be connected or running. This makes software development more user-friendly and flexible by allowing testing, debugging, and interaction with the full software suite in a controlled environment. All components of the software, including video streaming, tracking, and controls, are fully compatible with the development mode.
The requirements.txt file contains all the libraries used in this project:
- Numpy
- SciPy
- Matplotlib
- PyQt5 – Python bindings for the Qt application framework
- Qt Widgets - Qt Widgets module (via Qt/PySide or PyQt)
- QtPy - Abstraction layer for PyQt5/PyQt6/PySide2/PySide6
- OpenCV - OpenCV bindings for Python
- pyserial - Serial communication
- pygame - Joystick and controller support
- pipython - Python interface for Physik Instrumente (PI)
- pypylon - Python wrapper for Basler Pylon Camera Software Suite
- PyUSB - USB access library for Python
- PySocket - Python wrapper for socket communication
- Python 3.8 or higher
- Required Python packages listed in
requirements.txt(Installation Step 3)
The following commands are to be performed from your terminal on your local machine. Step 2 is not necessary but highly recommended.
-
Navigate to the directory on your local machine where you would like to save this repository using the following command:
cd your/local/path/ -
Creation and activation of a virtual environment with preferred method. The following commands show a simple example of how you may create and activate a virtual environment from your terminal. NAME is to be replaced with the desired name for the virtual environment. The first command creates the environment whereas the second command activates it.
python3.8 -m venv NAME
source NAME/bin/activatepython -m venv NAME
NAME\Scripts\activate
conda create -n NAME python=3.8
conda activate NAME
-
Run the following command in your terminal to install all the requirements:
pip install -r requirements.txt
-
Clone the repository:
git clone https://github.com/sinarollin/Catheter_Steering_Software.git
-
Go to the corresponding repository via your terminal (local or global):
cd Catheter_Steering_Software
[1] Microbiorobotics Systems Laboratory (MICROBS)
[2] Master Thesis: Teleoperation and Autonomous Navigation of a Flow-Driven Microcatheter for Retinoblastoma Treatment by Mehdi Krichen
[3] Julian Raub
[4] Gaël Gerber