Feature Request (a slightly big one) : add mujoco javascript/wasm support

[therealadityashankar]

Hey John!, not sure if you will have the time/patience for this, but mujoco now also supports javascript/wasm and has direct bindings to it, to have a variant of this that supports wasm would be a game changer for me ❤️

more info here https://github.com/google-deepmind/mujoco/tree/main/wasm

[johnsutor]

Hi @therealadityashankar, I am in the process of making the backend MuJoCo-only, so I am open to this request. What, in particular, do you envision the javascript/wasm bindings to look like? Would this be for teleop?

[therealadityashankar]

Hey John,
So this would be both for teleop and representation, I basically want something people can "directly run on the web" as a demo. I've used this repository for representation/demo work in the past, if this was extended to a js-binded wasm demo, I would be able to create a webapage through which people can try out models right from the browser, I also have a demo over here : https://robots.river.berlin/ where one can control a robot over the website, the project https://github.com/timpietrusky/lerobot.js/ also allows for people to use a so101 robot straight from javascript!

so this would allow people to see a demo of how a model works, while simultaneously being able to use it from js :)

[johnsutor]

@therealadityashankar apologies for the delay in response, wanted to clean up the repo post migrating to solely MuJoCo. I am trying to think of the best path forward to integrating this, and what to start with. I think it makes the most sense to ship a teleop version that is fully in-browser via WASM using the current Gradio UI that I have, if you think this would be a good first step.

[therealadityashankar]

Nothing to apologise for, I am honored that you are doing this >.<, ideally I'd like something that works with js on the longer run, via a cdn or npm - but for a start, wasm + gradio is awesome :)