[WIP] Fix ABI mismatch for nullable on-close callback

[Copilot]

Thanks for assigning this issue to me. I'm starting to work on it and will keep this PR's description up to date as I form a plan and make progress.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>fix(ffi): ABI mismatch for nullable on-close callback on System V platforms</issue_title>
<issue_description>## Summary

Passing Option<FnOnCloseCallback> (a type alias for extern "C" fn(...)) in extern "C" function signatures causes cbindgen to emit an opaque forward declaration instead of a nullable function pointer:

// cbindgen incorrectly generates:
typedef struct XmtpOption_FnOnCloseCallback XmtpOption_FnOnCloseCallback;

// instead of:
void (*on_close)(const char*, void*)

bindgen then translates this opaque struct into a zero-sized type (ZST) in Rust. When this ZST is passed by value across the FFI boundary, the two major x86_64 C ABIs handle it differently:

ABI	ZST handling	Result
System V (Linux, macOS)	Skipped entirely — no register or stack slot	All subsequent parameters (context, out) shift by 8 bytes
Windows MSVC x64	Occupies an 8-byte slot regardless of size	Parameters stay aligned; on_close value is garbage but context/out are correct

Symptoms

• Linux/macOS: Segmentation fault (core dumped) or fatal: xmtp ffi: null output pointer immediately after starting any stream.
• Windows: Appears to work correctly because the MSVC ABI preserves parameter alignment despite the ZST.

Root Cause

cbindgen cannot represent Option<TypeAlias> where TypeAlias is a function pointer type alias. It falls back to an opaque struct forward declaration. This is a known cbindgen limitation.

Affected Functions

All 6 stream functions in xmtp-ffi/src/stream.rs:

• xmtp_stream_conversations
• xmtp_stream_all_messages
• xmtp_conversation_stream_messages
• xmtp_stream_consent
• xmtp_stream_preferences
• xmtp_stream_message_deletions

Fix

Replace Option<FnOnCloseCallback> with the inline form in all extern "C" signatures:

// Before (broken):
on_close: Option<FnOnCloseCallback>,

// After (correct):
on_close: Option<unsafe extern "C" fn(*const c_char, *mut c_void)>,

This allows cbindgen to correctly generate a nullable function pointer in the C header. A module-level OnCloseCb type alias is used for internal Rust helpers.

Changes

• xmtp-ffi/src/stream.rs — inline Option<fn ptr> in 6 extern "C" signatures; add OnCloseCb alias with documentation
• xmtp-ffi/include/xmtp_ffi.h — regenerated by cbindgen (no manual edit)
• xmtp-sys/build.rs — defensive preprocessing to strip opaque forward declarations
• xmtp-sys/src/bindings.rs — regenerated by bindgen
• xmtp/src/stream.rs — remove no_on_close() helper, pass None directly

Prevention

Documented in xmtp-ffi/README.md and CONTRIBUTING.md:

Rule: In extern "C" function signatures, always use the inline Option<unsafe extern "C" fn(...)> form. Never use Option<FnMyCallback>.
</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes fix(ffi): ABI mismatch for nullable on-close callback on System V platforms #3

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Copilot]

Copilot wasn't able to review any files in this pull request.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.