Skip to content

Add PyQt HRI display prototype packageFeature/arlab UI pyqt package - #235

Merged
TobiasNeher merged 16 commits into
devfrom
feature/arlab-ui-pyqt-package
Aug 20, 2026
Merged

Add PyQt HRI display prototype packageFeature/arlab UI pyqt package#235
TobiasNeher merged 16 commits into
devfrom
feature/arlab-ui-pyqt-package

Conversation

@JayH7C3

@JayH7C3 JayH7C3 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Adds the PyQt-based HRI display prototype package for the UI part of Human-Robot Interaction.

The package implements a first ROS-integrated display prototype based on the previous UI mockup and addresses the open UI integration topics from Design Review 2. The old Tkinter demo UI is kept in legacy/, while the active prototype is implemented as a PyQt6 application.

The display can be started via ros2 run or the provided launch file and implements the following prototype pipeline:

Decision Making / Speech Output
-> publishes speech text on /tts_output
-> display subscribes to /tts_output
-> incoming messages are shown in the Speech Output card

User interaction
-> display buttons publish prototype feedback on /ui_action
-> current actions: confirm, repeat, cancel
-> this is intended as a temporary feedback path until a structured HRI feedback interface is defined

Camera display
-> display subscribes to /camera_gripper/color/image_raw
-> incoming ROS image messages are converted via cv_bridge
-> if no image topic is available, a local placeholder image is shown

The GUI provides:

  • Home screen for challenge/system mode selection
  • task-specific control views for Pick and Place and Laundry
  • Runtime View with camera, task status, speech output, next action and progress information
  • Developer View with runtime state and ROS/debug information

The current task/status/progress values are represented through an internal RuntimeState. This is used as a placeholder for the future structured HRI Output Interface from Decision Making.

Included:

  • ROS 2 package arlab_ui
  • executable zirbi_display
  • launch file zirbi_display.launch.py
  • PyQt6 GUI implementation
  • camera placeholder image
  • legacy Tkinter UI moved to legacy/

@JayH7C3
JayH7C3 requested a review from TobiasNeher July 22, 2026 17:43
@JayH7C3 JayH7C3 self-assigned this Jul 22, 2026
@JayH7C3 JayH7C3 added HRI Affects the Large Language Model Module of the system. Display HRI Display labels Jul 22, 2026

@TobiasNeher TobiasNeher left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • not tested on the actual zirbi hardware
  • the Legacy version was removed
  • code is commented
  • still a prototyp
    no further issues with the pyqt6 display implementation

@TobiasNeher
TobiasNeher merged commit 28c1d0c into dev Aug 20, 2026
1 check passed
@TobiasNeher
TobiasNeher deleted the feature/arlab-ui-pyqt-package branch August 20, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Display HRI Display HRI Affects the Large Language Model Module of the system.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants