Skip to content

Grow the guest heap through the store, not around it - #61

Open
Arshia001 wants to merge 2 commits into
mainfrom
fix/guest-heap-store-lending
Open

Grow the guest heap through the store, not around it#61
Arshia001 wants to merge 2 commits into
mainfrom
fix/guest-heap-store-lending

Conversation

@Arshia001

Copy link
Copy Markdown
Member

Needs the paired wasmer change: wasmerio/wasmer#6887 (stacked on #6886). It will not build against wasmer main until those land.

Why

GuestHeap grew guest linear memory through a cloned VMSharedMemory, obtained from a SharedMemory::vm_shared_memory() accessor that existed for this one caller. Growing a memory outside the store that owns it is not an operation the wasmer API should have, and it is being dropped. wasmer instead grew a way to hand the store to code that cannot be passed one.

The bind this is in: the allocation that needs the growth arrives with no store in sight. V8 calls the array-buffer allocator from inside its own frames, several levels below the N-API import that entered it, and no Rust reference survives that trip.

What changed

The import lends its store across the boundary — StoreMut::parked — and the heap picks it back up with Store::with_current, then grows through the ordinary Memory::grow. Lending happens in with_callback_state, which already wraps every bridge call that can run JS, plus the handful of import handlers that allocate directly (via the new alloc_guest helper).

Fallout worth knowing about:

  • The shared/owned split is gone. Any heap grows when a store is lent and falls back to its arena when none is — which is what a V8 background thread will find. So every heap is now pre-funded and topped up from import boundaries, not just the non-shared ones.
  • Handles are store-scoped. WASIX worker threads share one linear memory but each has its own store, so the heap keeps one handle per store that reaches it and grows through the one belonging to the store being lent. get_or_create registers the caller's.
  • The style check reads MemoryStyle off the engine's tunables instead of the VM memory, so the file no longer reaches into wasmer::sys::vm at all.
  • Pre-funding is best effort. An unfunded heap is slower, not broken, since any allocation made with a store lent can claim for itself.

callback.rs moves with it. Its callback context held a raw pointer to the installing frame's FunctionEnvMut — which is precisely the borrow the import now parks, and reaching around a park would defeat the point of it. It holds a store-free FunctionEnv handle instead and picks the store up the same way the heap does. That removes the last raw-pointer smuggling from the file.

rust-toolchain.toml goes to 1.95, which wasmer's MSRV now requires.

Testing

40/40 lib tests. New ones cover: an allocation served from the arena with no store lent at all; the same allocation past the arena failing rather than growing the guest's memory behind its back; the same request succeeding once a store is lent; and a store that never registered a handle being unable to grow the memory.

`GuestHeap` grew guest linear memory through a cloned `VMSharedMemory`,
taken from a `SharedMemory::vm_shared_memory()` accessor that existed for
this one caller. Growing a memory outside the store that owns it is not an
operation the wasmer API should have, so it is going away; wasmer instead
grew a way to hand the store to code that cannot be passed one
(wasmerio/wasmer#6887).

The bind is that the allocation which needs the growth arrives with no
store in sight: V8 calls the array-buffer allocator from inside its own
frames, several levels below the N-API import that entered it. So the
import lends its store across that boundary — `StoreMut::parked` — and the
heap picks it back up with `Store::with_current` and grows through the
ordinary `Memory::grow`. The lending is in `with_callback_state`, which
already wraps every bridge call that can run JS, plus the handful of import
handlers that allocate directly (through `alloc_guest`).

Consequences worth knowing:

  * The shared/owned split is gone. Any heap grows when a store is lent,
    and any heap falls back to its arena when none is — which is the case
    for a V8 background thread, so every heap is now pre-funded and topped
    up from import boundaries, not just the non-shared ones.
  * Memory handles are store-scoped, and WASIX worker threads share one
    linear memory while each has its own store, so the heap keeps one
    handle per store that reaches it and grows through the one belonging to
    the store being lent. `get_or_create` registers the caller's.
  * The style check reads `MemoryStyle` from the engine's tunables rather
    than from the VM memory, so the file no longer reaches into
    `wasmer::sys::vm` at all.
  * Pre-funding is best effort now. An unfunded heap is slower, not broken,
    since any allocation made with a store lent can claim for itself.

`callback.rs` changes with it: the callback context held a raw pointer to
the installing frame's `FunctionEnvMut`, which is exactly the borrow the
import now parks — reaching around a park would defeat it. It holds a
store-free `FunctionEnv` handle instead and picks the store up the same way
the heap does, so the file has no raw-pointer smuggling left.

Also bumps rust-toolchain.toml to 1.95, which wasmer's MSRV now requires.

Tested: 40/40 lib tests, including new ones covering growth with and
without a lent store, and a store that never registered a handle.
Arshia001 added a commit to wasmerio/wasmer that referenced this pull request Aug 14, 2026
wasmer-napi's `GuestHeap` grew guest linear memory through a cloned
`VMSharedMemory`, reached by an accessor added to wasmer for that one
caller. Growing a memory outside the store that owns it does not belong in
the API, and the previous attempt at this (#6877) kept the operation and
merely gave it a tidier signature.

With `StoreMut::parked` / `Store::with_current` in place, the heap does not
need one: the N-API import lends its store across the bridge call, and the
V8 allocator hook — several C++ frames below it, where no Rust reference
survives — picks it back up and grows through the ordinary `Memory::grow`.
No store-free operation is left on either side.

Pins wasmerio/napi#61, and picks up the `offset-allocator`/`nonmax` lock
entries its allocator needs. The pin moves to napi `main` once that merges.
The consuming Wasmer checkout lints its submodules with taplo, so
unformatted manifests here fail `make lint` there.
Arshia001 added a commit to wasmerio/wasmer that referenced this pull request Aug 14, 2026
wasmer-napi's `GuestHeap` grew guest linear memory through a cloned
`VMSharedMemory`, reached by an accessor added to wasmer for that one
caller. Growing a memory outside the store that owns it does not belong in
the API, and the previous attempt at this (#6877) kept the operation and
merely gave it a tidier signature.

With `StoreMut::parked` / `Store::with_current` in place, the heap does not
need one: the N-API import lends its store across the bridge call, and the
V8 allocator hook — several C++ frames below it, where no Rust reference
survives — picks it back up and grows through the ordinary `Memory::grow`.
No store-free operation is left on either side.

Pins wasmerio/napi#61, and picks up the `offset-allocator`/`nonmax` lock
entries its allocator needs. The pin moves to napi `main` once that merges.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant