perf: reuse a thread-local scratch buffer for new_from_utf8 transcode - #2027
perf: reuse a thread-local scratch buffer for new_from_utf8 transcode#2027bartlomieju wants to merge 5 commits into
Conversation
c79b84f to
5faa602
Compare
V8's NewFromUtf8 runs a scalar UTF-8 decoder (twice: once for width/length, once to write), which is extremely slow for non-ASCII input. When simdutf is available, decode the input ourselves and hand V8 a pre-decoded buffer it can memcpy: - Pure ASCII -> NewFromOneByte directly (bytes are already valid Latin-1). - Non-ASCII, >= 16 bytes -> try convert_utf8_to_latin1_with_errors (one-byte) then convert_utf8_to_utf16le_with_errors (two-byte); on invalid UTF-8, fall back to V8's lossy NewFromUtf8. - Tiny non-ASCII (< 16 bytes) stays on V8's decoder (the two simdutf FFI calls cost more there). - Inputs whose byte length exceeds MAX_LENGTH are left to V8, which rejects them (preserving the conservative None behavior; our shorter decoded length would otherwise accept some). new_from_utf8 (median of 5): ascii: -25% (4) .. -10% (256), ~0 at 4096 latin1: -44% (16), -75% (64), -87% (256), -93% (4096) twobyte: -33% (16), -74% (64), -88% (256), -93% (4096) Non-ASCII string creation (accented text, CJK, emoji, unicode JSON) is up to ~14x faster. ASCII and short non-ASCII are unaffected/faster. Adds a test for the transcode paths + invalid-UTF-8 fallback and benches/string_encode.rs. Supersedes #2020 (which was based on a stale main and did only the ASCII half with std is_ascii).
Match benches/function.rs: a harness=false bench must produce no output when nextest enumerates test binaries.
…ranscode The simdutf transcode path in new_from_utf8 allocated a fresh Vec on every non-ASCII string creation. Reuse a thread-local scratch buffer instead (taken out and put back around the V8 call so its borrow isn't held across a possible GC). The buffer is copied into V8 by NewFromOneByte/NewFromTwoByte as before, so only the per-call allocation is removed. Incremental over #2026 for non-ASCII creation: latin1: -13% (16), -13% (64), -15% (256), -3% (4096) twobyte: -24% (16), -24% (64), -18% (256), -3% (4096) The allocation was the dominant fixed cost at small-medium sizes. (This is the low-risk alternative to a NewRawOneByte/TwoByte binding that would also eliminate the copy: that only helps large strings by a few percent and needs unsafe V8-internal Factory access + GC-safety care, so it is left as a possible follow-up.) Stacked on #2026.
be7b918 to
cac1895
Compare
5faa602 to
fb5a4a8
Compare
…atch # Conflicts: # src/string.rs
|
This needs to be rewritten to have a limit on size - otherwise allocating an 100Mb string will permamently leave 100Mb laying around |
Address review: the thread-local scratch grew to the input length and was put back uncapped, so a one-off huge string (e.g. 100 MB) left that allocation live in the thread-local forever. Drop capacity back to ENCODE_SCRATCH_MAX_CAP (64Ki elements = 64 KiB Latin-1 / 128 KiB UTF-16) on put-back via put_encode_scratch; a larger string still transcodes, it just reallocates that one time. Extends the transcode test with >cap Latin-1/UTF-16 strings plus a small string afterwards to cover the shrink-then-reuse path.
|
Addressed in 19eac93. The scratch buffer's retained capacity is now capped: on put-back, The cap sits comfortably past the sizes where reuse actually helps — per the PR's own numbers the allocation saving has faded to ~−3% by 4 KiB and is dominated by the copy/transcode well before 64 KiB. Extended |
Summary
#2026's simdutf transcode path in
new_from_utf8allocated a freshVeconevery non-ASCII string creation. This reuses a thread-local scratch buffer
instead — taken out of the thread-local and put back around the V8 call, so its
borrow isn't held across a potential GC. The buffer is still copied into V8 by
NewFromOneByte/NewFromTwoByte, so only the per-call allocation is removed.Impact
Incremental over #2026,
new_from_utf8(median of 6):The allocation was the dominant fixed cost at small-medium sizes; it fades
against the copy/transcode for large strings.
Note on the alternative
This is the low-risk half of proposal "3B". A
NewRawOneByteString/NewRawTwoByteStringbinding could additionally eliminate the copy bytranscoding straight into V8's heap string, but that only helps large strings
by a few percent and requires unsafe V8-internal
Factoryaccess + writingunder a
DisallowGarbageCollectionscope, so it's left as a possible follow-up.Correctness
All 16
test_apistring tests pass (with and withoutsimdutf).