Exposing asynchronous functions to Julia

[lenianiva]

Is it possible to expose Rust async functions to Julia directly? What kind of async primitives can we access in this context?

[Taaitaaiger]

Async functions are currently only supported for embedding with the async runtime. BackgroundTasks and DelegatedTasks exist, but I have to admit they've barely been tested.

[lenianiva]

Async functions are currently only supported for embedding with the async runtime. BackgroundTasks and DelegatedTasks exist, but I have to admit they've barely been tested.

so if the Rust part is a ccall library it cannot expose any async functions? What if the rust part spins up its own runtime to run async functions, but only exposes sync interfaces?

I checked the example in the unit tests and the source code for BackgroundTask and DelegatedTask. Both of these functions spawn their own threads and do not directly interact with the Rust async interface. However, they could be adapted to communicate with an existing async runtime. Maybe the best solution here is to spin up a thread pool when the library gets loaded, and communicate with this thread pool using BackgroundTasks and DelegatedTasks.

In this case, any Julia object sent to this pool of executors must implement the Send trait. I think this holds for Value but I'm not entirely sure.

[Taaitaaiger]

The main issue is that the async runtime assumes it's responsible for initializing Julia, and the same is true for the combined multithreaded / async runtime. There's also a few additional challenges (when a runtime feature is enabled, JULIA_FAST_TLS is enabled which is UB in a shared library. I also expect challenges with GC-safety.)

If you just want to call async Rust code on a separate thread and not call back into Julia, it should be okay to start a tokio runtime on that thread.

[lenianiva]

The main issue is that the async runtime assumes it's responsible for initializing Julia, and the same is true for the combined multithreaded / async runtime. There's also a few additional challenges (when a runtime feature is enabled, JULIA_FAST_TLS is enabled which is UB in a shared library. I also expect challenges with GC-safety.)

If you just want to call async Rust code on a separate thread and not call back into Julia, it should be okay to start a tokio runtime on that thread.

What if I pass a callback function from the Julia side into the Rust side? Would I need to register the threads in the pool?

[Taaitaaiger]

Using such pools is only supported when embedding, they can't be used from exported functions (unless you use some pointer shenanigans).

Could you tell me more about what you're trying to achieve? I feel like I can do a better job of explaining the possibilities and trade-offs if I know that.

[lenianiva]

Using such pools is only supported when embedding, they can't be used from exported functions (unless you use some pointer shenanigans).

Could you tell me more about what you're trying to achieve? I feel like I can do a better job of explaining the possibilities and trade-offs if I know that.

What would I have to do to register threads spawned from Rust with Julia? Is something like this sufficient? Another method is what I read in the manual about thread safety, where the callee should schedule calls with DelegatedTask.

I have a search algorithm in Rust, and it is mostly async. The search algorithm takes an agent and calls functions on the agent to direct the search. I want to supply the agent from the Julia side.

[Taaitaaiger]

In addition to adopting the thread, you must guarantee that the thread is made GC-safe before exiting. Any code that doesn't call into Julia for extended periods of time must also be gc-safe to avoid blocking the GC and potential deadlocks.

You must also take care that the thread exits before the main Julia thread does (or is otherwise shut down). Adopted threads call a "destructor" when they exit, and this can deadlock if Julia is no longer active.

The part in the manual matches what BackgroundTasks and DelegatedTasks do internally: they contain a Base.AsyncCondition and wait on it before accessing and returning the result of the task.

[lenianiva]

In addition to adopting the thread, you must guarantee that the thread is made GC-safe before exiting. Any code that doesn't call into Julia for extended periods of time must also be gc-safe to avoid blocking the GC and potential deadlocks.

You must also take care that the thread exits before the main Julia thread does (or is otherwise shut down). Adopted threads call a "destructor" when they exit, and this can deadlock if Julia is no longer active.

The part in the manual matches what BackgroundTasks and DelegatedTasks do internally: they contain a Base.AsyncCondition and wait on it before accessing and returning the result of the task.

Thanks! I'll implement a runtime like this

[Taaitaaiger]

No worries, feel free to open a new issue if you have any questions!

[lenianiva]

No worries, feel free to open a new issue if you have any questions!

Would it be possible to make

jlrs/crates/jlrs/src/runtime/handle/mt_handle/mod.rs

Line 197 in 0e553aa

unsafe fn adopt_thread() {

and other thread handling functions public so the async library can call them? Does the async runtime need to use functions in jlrs-sys and jl-sys?

[Taaitaaiger]

I don't think that function should be exported, as it would require exporting a bunch of implementation details. I think it would make more sense to export a stand-alone version of MtHandle::with and mark it unsafe because it can be called from threads that potentially outlive the main Julia thread:

    pub unsafe fn with<T>(func: impl for<'ctx> FnOnce(Pin<&'ctx mut WeakHandle>) -> T) -> T {
        unsafe {
            if !ADOPTED.get() {
                adopt_thread();
            }

            gc_unsafe(|_| {
                let weak = weak_handle_unchecked!();
                func(weak)
            })
        }
    }

[lenianiva]

I don't think that function should be exported, as it would require exporting a bunch of implementation details. I think it would make more sense to export a stand-alone version of MtHandle::with and mark it unsafe because it can be called from threads that potentially outlive the main Julia thread:

pub unsafe fn with<T>(func: impl for<'ctx> FnOnce(Pin<&'ctx mut WeakHandle>) -> T) -> T {
    unsafe {
        if !ADOPTED.get() {
            adopt_thread();
        }

        gc_unsafe(|_| {
            let weak = weak_handle_unchecked!();
            func(weak)
        })
    }
}

and this would be called with static lifetimes on both parameters?

Currently I just do

/* thread starts */
let _pgc = unsafe { jl_adopt_thread() };
/* thread terminates */
let ptls = unsafe { jlrs_get_ptls_states() };
unsafe { jlrs_gc_safe_enter(ptls) };

and it seems to be fine...

[Taaitaaiger]

Do you mean the 'ctx lifetime? It's a higher-kinded lifetime is used to express that the WeakHandle cannot be returned from the closure (as that would allow calling into Julia from a GC-safe state)

Your approach is fine if you can guarantee it will only be called once, but adopting a thread a second time is UB. My approach avoids that footgun.