Grow the guest heap through the store, not around it - #61
Open
Arshia001 wants to merge 2 commits into
Open
Conversation
`GuestHeap` grew guest linear memory through a cloned `VMSharedMemory`, taken from a `SharedMemory::vm_shared_memory()` accessor that existed for this one caller. Growing a memory outside the store that owns it is not an operation the wasmer API should have, so it is going away; wasmer instead grew a way to hand the store to code that cannot be passed one (wasmerio/wasmer#6887). The bind is that the allocation which needs the growth arrives with no store in sight: V8 calls the array-buffer allocator from inside its own frames, several levels below the N-API import that entered it. So the import lends its store across that boundary — `StoreMut::parked` — and the heap picks it back up with `Store::with_current` and grows through the ordinary `Memory::grow`. The lending is in `with_callback_state`, which already wraps every bridge call that can run JS, plus the handful of import handlers that allocate directly (through `alloc_guest`). Consequences worth knowing: * The shared/owned split is gone. Any heap grows when a store is lent, and any heap falls back to its arena when none is — which is the case for a V8 background thread, so every heap is now pre-funded and topped up from import boundaries, not just the non-shared ones. * Memory handles are store-scoped, and WASIX worker threads share one linear memory while each has its own store, so the heap keeps one handle per store that reaches it and grows through the one belonging to the store being lent. `get_or_create` registers the caller's. * The style check reads `MemoryStyle` from the engine's tunables rather than from the VM memory, so the file no longer reaches into `wasmer::sys::vm` at all. * Pre-funding is best effort now. An unfunded heap is slower, not broken, since any allocation made with a store lent can claim for itself. `callback.rs` changes with it: the callback context held a raw pointer to the installing frame's `FunctionEnvMut`, which is exactly the borrow the import now parks — reaching around a park would defeat it. It holds a store-free `FunctionEnv` handle instead and picks the store up the same way the heap does, so the file has no raw-pointer smuggling left. Also bumps rust-toolchain.toml to 1.95, which wasmer's MSRV now requires. Tested: 40/40 lib tests, including new ones covering growth with and without a lent store, and a store that never registered a handle.
Arshia001
added a commit
to wasmerio/wasmer
that referenced
this pull request
Aug 14, 2026
wasmer-napi's `GuestHeap` grew guest linear memory through a cloned `VMSharedMemory`, reached by an accessor added to wasmer for that one caller. Growing a memory outside the store that owns it does not belong in the API, and the previous attempt at this (#6877) kept the operation and merely gave it a tidier signature. With `StoreMut::parked` / `Store::with_current` in place, the heap does not need one: the N-API import lends its store across the bridge call, and the V8 allocator hook — several C++ frames below it, where no Rust reference survives — picks it back up and grows through the ordinary `Memory::grow`. No store-free operation is left on either side. Pins wasmerio/napi#61, and picks up the `offset-allocator`/`nonmax` lock entries its allocator needs. The pin moves to napi `main` once that merges.
The consuming Wasmer checkout lints its submodules with taplo, so unformatted manifests here fail `make lint` there.
Arshia001
added a commit
to wasmerio/wasmer
that referenced
this pull request
Aug 14, 2026
wasmer-napi's `GuestHeap` grew guest linear memory through a cloned `VMSharedMemory`, reached by an accessor added to wasmer for that one caller. Growing a memory outside the store that owns it does not belong in the API, and the previous attempt at this (#6877) kept the operation and merely gave it a tidier signature. With `StoreMut::parked` / `Store::with_current` in place, the heap does not need one: the N-API import lends its store across the bridge call, and the V8 allocator hook — several C++ frames below it, where no Rust reference survives — picks it back up and grows through the ordinary `Memory::grow`. No store-free operation is left on either side. Pins wasmerio/napi#61, and picks up the `offset-allocator`/`nonmax` lock entries its allocator needs. The pin moves to napi `main` once that merges.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Needs the paired wasmer change: wasmerio/wasmer#6887 (stacked on #6886). It will not build against wasmer
mainuntil those land.Why
GuestHeapgrew guest linear memory through a clonedVMSharedMemory, obtained from aSharedMemory::vm_shared_memory()accessor that existed for this one caller. Growing a memory outside the store that owns it is not an operation the wasmer API should have, and it is being dropped. wasmer instead grew a way to hand the store to code that cannot be passed one.The bind this is in: the allocation that needs the growth arrives with no store in sight. V8 calls the array-buffer allocator from inside its own frames, several levels below the N-API import that entered it, and no Rust reference survives that trip.
What changed
The import lends its store across the boundary —
StoreMut::parked— and the heap picks it back up withStore::with_current, then grows through the ordinaryMemory::grow. Lending happens inwith_callback_state, which already wraps every bridge call that can run JS, plus the handful of import handlers that allocate directly (via the newalloc_guesthelper).Fallout worth knowing about:
get_or_createregisters the caller's.MemoryStyleoff the engine's tunables instead of the VM memory, so the file no longer reaches intowasmer::sys::vmat all.callback.rsmoves with it. Its callback context held a raw pointer to the installing frame'sFunctionEnvMut— which is precisely the borrow the import now parks, and reaching around a park would defeat the point of it. It holds a store-freeFunctionEnvhandle instead and picks the store up the same way the heap does. That removes the last raw-pointer smuggling from the file.rust-toolchain.tomlgoes to 1.95, which wasmer's MSRV now requires.Testing
40/40 lib tests. New ones cover: an allocation served from the arena with no store lent at all; the same allocation past the arena failing rather than growing the guest's memory behind its back; the same request succeeding once a store is lent; and a store that never registered a handle being unable to grow the memory.