Project: Collaborative AR application using OpenIGTLink, 3D Slicer and Magic Leap 2

[AliciaPoseDiezdelaLastra]

Draft Status

Ready - team will start page creating immediately

Category

VR/AR and Rendering

Key Investigators

• Alicia Pose Díez de la Lastra (Universidad Carlos III de Madrid, Madrid, Spain)
• Javier Pascau (Universidad Carlos III de Madrid, Madrid, Spain)
• Steve Pieper (Isomics, USA)
• Ron Kikinis (BWH, USA)

Project Description

In previous Project Weeks, we showed a way to communicate Microsoft HoloLens 2 with 3D Slicer using OpenIGTLink protocol. The current solution is implemented in a 3 elements system. It is composed by A Microsoft HoloLens 2 headset, the Unity software, and the 3D Slicer platform. The HoloLens 2 application is not directly built on the device, but remotely transferred from Unity in real time using Holographic Remoting. The communication workflow is represented in the following image:

The results of that work are publicly available at this GitHub repository.

In more recent Project Weeks we evaluated the transferability of the aforementioned project to other AR devices. Specifically, we focused on the VARJO XR-3 headset, obtaining fast and high-quality results:

RealTimeMessageTransfer.mp4

CurtisTest.mp4

This time, we aim to follow a similar approach, this time with Magic Leap 2, with the ultimate goal of creating a colaborative application between multiple headsets communicating through OpenIGTLink.

Objective

1. Objective A: Adapt previous Unity apps to Magic Leap 2.
2. Objective B: Exchange information between Magic Leap 2 and 3D Slicer.

Approach and Plan

1. Adapt previous Unity apps to Magic Leap 2.
2. Test Holographic Remoting in Magic Leap 2.
3. Test the exchange of information between Magic Leap 2 and 3D Slicer.
4. Add a new headset to the system and try to exchange information between HoloLens 2 and Magic Leap 2

Progress and Next Steps

1. Bring both a HoloLens 2 and a Magic Leap 2 headsets to PW.

Illustrations

No response

Background and References

No response

[github-actions]

Project Page Pull Request Creation

✅ COMPLETED: See #1628

[AliciaPoseDiezdelaLastra]

I'd like to add both Steve Pieper and Ron Kikinis as Key Investigators of the project.
Thank you!