Multilanguage - FFI

[Murmele]

This discussion shall be used as base for interfaces for other languages.

For Rust:

• https://github.com/maplibre/maplibre-native-rs.git
• Initial interfacing
• Sharing Textures: using custom created WebGPU implementation based on wgpu. PR Pending
• Android. Working prototype

Faced problems:

• Sharing Textures. The existing wgpu-native was not suitable because it builds on wgpu-core and creating a wgpu Texture from it is not possible (at least to my knowledge)
• Android: special platform interface required because only ndk is used
  • setting jvm from Rust
  • Creating http declarations in maplibre-native, implementations done in Rust

For the platform interface I started creating a new "platform". Below is a prototype for the http bridge to gain internet access:

• maplibre-native side (Declaration and call): https://github.com/Murmele/maplibre-native/tree/android/platform/rust/src
• maplibre-native-rs side (Implementation): https://github.com/Murmele/maplibre-native-rs/blob/199c248214a64b5cb3fcf5445cecd08490eca9ab/src/http_bridge.rs

Just write down your thoughts and your problems you faced to interface maplibre-native
What are your approaches to get for example internet access?

[sargunv]

Main problem for MapLibre Compose was the varying API surfaces across MapLibre Native Android and iOS, plus the complexity of integrating with the C++ core (from JVM for desktop, plus eventually Android JNI / Kotlin Native).

So https://github.com/maplibre/maplibre-native-ffi exposes one uniform C ABI, as C is easily callable from many languages/runtimes. So the idea is I write direct bindings to that C API for languages that can import them (Java FFM, Zig, Rust, Swift, Kotlin, ...), and adapters through Rust for those that need native-side glue (Node, Python, Java JNI).

(it would also be reasonable to invert the C/Rust relationship here, with Rust bound to MLN via cxx like you did and a #[no_mangle] pub extern fn C adapter on Rust for those languages that need it)

In my cmake build I build MLN_WITH_CORE_ONLY and explicitly list the additional files and libraries MLN requires for the platform, so I get deep control over each platform's build.

Currently supports:

• all MLN functionality except for deep extension stuff like custom layers and shader registry
• macOS Metal/Vulkan and Linux Vulkan. (builds on windows too but not yet functional)
• Native surfaces, shared and owned textures, CPU readback with owned textures
• Java FFM bindings with an lwjgl example app
• Examples written in (very rough) zig and swift using the raw C api

The existing wgpu-native was not suitable because it builds on wgpu-core and creating a wgpu Texture from it is not possible (at least to my knowledge)

I haven't tried this with wgpu yet, but sharing textures works well with Metal and Vulkan at least.

Creating http declarations in maplibre-native, implementations done in Rust

Curious, is there a reason whatever http implementation MLN already uses for Android wasn't usable in your Rust wrapper?

[Murmele]

Thanks for the description. I am building also only the mbgl-core target with cmake in the build script.

For Android I didn't dig too much into it. Maybe using maplibre target instead of mbgl-core would be easier because it contains all the required files and the registration of the java classes?

I have seen the vulkan backend contains a unique_ptr to a Texture2D are you moving this out and create a new one to share the textures?

maplibre-native/platform/default/include/mbgl/vulkan/headless_backend.hpp

Line 42 in 9faee47

std::unique_ptr<Texture2D> texture;

[sargunv]

I have two texture modes:

• Owned texture session: MapLibre creates the Vulkan image, then exposes borrowed handles via mln_vulkan_owned_texture_acquire_frame().
• Borrowed texture session: the host passes its own VkImage/VkImageView to mln_vulkan_borrowed_texture_attach(), and MapLibre renders directly into that image.

I have my own VulkanTextureBackend, which implements mbgl::gfx::HeadlessBackend and mbgl::vulkan::RendererBackend, so I'm not using that mbgl::vulkan::HeadlessBackend. So my backend exposes a temporary borrowed view of the texture for the C API through mln_vulkan_owned_texture_acquire_frame() and ..._release_frame().

This does mean rendering is blocked while the caller is sampling the texture; it'd be better to offer a swapchain option to keep things flowing (at the cost of latency), tracked in maplibre/maplibre-native-ffi#138.

[Murmele]

I don't have this problem because I move out the texture. But instead of returning the used texture so it could be reused, I just discard it and create in maplibre-native a new one which is also not idea yet