Bump napi: fix the V8+WASIX large-payload leak and the host-heap corruption abort - #143
Open
Arshia001 wants to merge 4 commits into
Open
Bump napi: fix the V8+WASIX large-payload leak and the host-heap corruption abort#143Arshia001 wants to merge 4 commits into
Arshia001 wants to merge 4 commits into
Conversation
…o the chunk Fixes the dominant term in the V8+WASIX large-payload leak — each 1 MiB guest chunk was costing ~2 MiB of permanently-retained host allocator metadata.
Picks up the fix for the host-heap corruption that aborted edgejs-v8 with "malloc_consolidate(): unaligned fastbin chunk detected" under sustained load: V8's ArrayBufferSweeper background threads were mutating the env's hint unordered_set concurrently with the main thread.
… handle Picks up wasmerio/napi#54, which replaces the private-VM-type accessor GuestHeap was using with the store-free SharedMemory::grow/data_ptr/style API added in wasmerio/wasmer#6877. Removes an unwrap_sys_ref() that would panic on the v8/js backends, and the last unsafe deref on the grow path.
wasmerio/wasmer#6848 ("port to vanilla Wild and use in-memory FS") removed the lib/wild submodule. provision-wasmer.sh asked for it unconditionally, so checking out any wasmer ref at or after that commit died with error: pathspec 'lib/wild' did not match any file(s) known to git before a single thing was built, taking every lane that provisions wasmer with it. Guard the init on the path actually being in the tree, which keeps older refs (which do need it, since wasmer-compiler path-depends on wasmer-wild at lib/wild/libwild) working unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Submodule bumps only. Picks up three napi changes covering two independent bugs on the V8/N-API WASIX lane plus the API cleanup that came out of reviewing them. quickjs is unaffected by any of it.
1. Guest-heap chunk allocators leaked ~2 MiB of host metadata per 1 MiB chunk (wasmerio/napi#53). Every
GuestHeapchunk built its offset allocator withAllocator::new, which sizes the node arena for 128Ki allocations no matter how small the chunk is. Chunks are claimed as a workload's peak grows and never released, so host RSS grew about three times the guest memory being managed. On the payload load test (echo-buffer, 10 MiB bodies, 1.95 GiB moved), net RSS growth drops 368 MB -> 134 MB and host-anonymous growth drops ~20 MB/GiB -> ~1 MB/GiB.2. The external-backing-store hint set was mutated from V8's ArrayBufferSweeper background threads (wasmerio/napi#53) with no synchronization against the main thread, corrupting the host heap. The runtime aborted with
malloc_consolidate(): unaligned fastbin chunk detecteda few hundred requests into a payload-heavy workload. With a config that reproduces it, 2 of 6 runs died before the fix and 0 of 28 after.3.
GuestHeapno longer reaches into wasmer's private VM types (wasmerio/napi#54). It was holding aVMSharedMemoryobtained through an accessor that returned an unnameable type and panicked on non-sys backends; it now uses the store-freeSharedMemory::grow/data_ptr/styleAPI added in wasmerio/wasmer#6877.Merge order
This branch's napi pin is the head of wasmerio/napi#54, so:
wasmer run edgejs.wasmcrashes when exitting #54's head too)wasmerio/wasmer#6877 replaces the previous ten-PR
chore: bump napistack (#6819-#6850), which has been closed.Testing
Verified locally against this branch's napi pin, built into a
wasmerCLI with--features napi-v8,llvm:wasmer-napilib tests green (2 new).Note for reviewers
For the
v8-wasixCI lane to actually exercise any of this, theWASMER_SOURCE_REFrepo var needs to point at a wasmer build carrying wasmerio/wasmer#6877 — otherwise the lane builds the runtime with wasmer main's older napi pin and tests none of these fixes.