Apple Vision Pro Support

[BenArtes]

I see you started a Quest3 port. I'd like to work on a port for Apple Vision Pro.

What's the status of the Quest3 port and do you have any thoughts on the best branch to start from (I assume quest_3_updates).

[kodie-artner]

Hi Ben, yeah the Quest3 branch would be the best to start from. Let me gather some thoughts on this and I'll respond again tomorrow.

[kodie-artner]

@BenArtes
I would start with the quest 3 branch. I bought a house midway though developing and haven't had any free time since. The current state of the quest 3 branch is very basic. I added some support for controllers and I updated the ui to be attached to said controller.
I went off trying to add support for web browsers and ssh terminals but got blocked on both ends. I should limit the scope to basic functionality.

First thing to look into is the vision pro controls. Does the XR Interaction kit support the vision pro? If so, that makes it a lot easier as I have already added that for the quest 3.
After that, you would just need to update the settings page and find some adequate controls for the vision pro to use.
I am highly interested in seeing this working on the Quest 3 and Vision Pro so I am happy to help you in anyway I can, answering questions about current architecture, what the interfaces should look like, etc. So just reach out. Thanks!

[BenArtes]

Okay, those are great starting points. I'm also very excited for AR views of robots overlayed on the passthrough. I think it will drastically improve insight/debugging. Been thinking about it for 1-2 years, so I was very excited when I stumbled on AR-RViz.

I got the app building for iOS devices. I started looking at the quest_3_update branch and it requires Unity 2023 and AR Foundation 6; unfortunately it looks like Unity hasn't released visionOS for 2023, only 2022. So I'm kinda on pause until they update that.

In good news, it does look like the XR Interaction Kit supports vision pro, so that should be straightforward.

At the same time, I have a decent amount to learn about creating various types of applications for visionOS in Unity and getting them to generate correctly through XCode (specifically Mixed/Immersive apps).

While I wait for unity to update visionOS to 2023, I'll probably disappear for a bit and poke around creating those types of apps from scratch, but I'll poke this again in a few weeks when I've hopefully made some progress.

[kodie-artner]

The quest 3 branch should be on 2022.3.12f1 I updated to the latest LTS version because the quest 3 required it.

The great thing about Unity is that you don't even need to touch Xcode. All the Xcode stuff is generated automatically. The Vision OS does vary a bit from normal AR/VR headsets though so I do agree, that building some basic vision pro apps would be a good first step.
Do you have a vision pro? And a robot running ROS? My biggest blocker at the moment is that I am trying to set up a robot in simulation to test all the features but decided to use Isaac sim to test it out too and it is taking longer than I'd like to get working with Nav2.

But yeah, take your time. A few weeks is like a day in open-source land haha

[BenArtes]

Oh, I have been manually building in XCode lol. Good to know.

I do have an AVP and a few robots at work to test this out with. Once I have something up and running I may even be able to convince my employer to let me spend some company time working on this.

And yeah, I tried Isaac sim (really wanted it to work), but after a few days had to move on because we had other stuff going on. The Omniverse stuff was weirdly difficult to get up and running.

[kodie-artner]

Yeah, "build and run" is the command you want to use in Unity!
I started a slack workspace to communicate a bit easier because someone else is interested in working on the quest 3 side. https://join.slack.com/t/ar-rviz/shared_invite/zt-2c71pccb6-Y_EA9YeuwtpH9BzU~r77dg

Okay great, having those to test with will make everything so much easier. Do you have Unity Pro? Looks like you need it to use the vision pro sdk.
If you can't tell, I am very excited you are interested in helping out!