Skip to content

fix(cranelift): publish the caller's rwasm counters before a tail call - #15

Merged
dmitry123 merged 2 commits into
v45-patchedfrom
fix/rwasm-stack-tail-call
Sep 28, 2026
Merged

dmitry123 merged 2 commits into
v45-patchedfrom
fix/rwasm-stack-tail-call

Conversation

@dmitry123

Copy link
Copy Markdown
Member

Summary

  • The rwasm stack-limit emulation (feat(cranelift): emulate the rwasm stack limits in compiled code #13) keeps one packed counter (call depth, frame base) in the store. A plain call publishes the callee's values and nothing restores the caller's afterwards; a tail call published nothing at all. So a frame that made a plain call and then a return_call handed the tail callee the callee's counters, and every iteration of a tail-recursive loop with a call in its body counted as one more frame until StackOverflow at N_MAX_RECURSION_DEPTH. The rwasm VM runs the same module, so the two backends diverged.
  • return_call, return_call_indirect and return_call_ref now store the caller's own counters (the SSA value its prologue loaded) before the call, like ReturnCallInternal. One extra store per tail-call site; modules without tail calls compile to the same code.
  • Version 45.0.0-rwasm.4 for the three published crates.

Reproduction

rwasm tests/stack-overflow.rs, new case tail_calls_after_a_call_do_not_count_towards_the_recursion_limit (return_call and return_call_indirect after a call, 100 × N_MAX_RECURSION_DEPTH iterations):

  • rwasm.3 from crates.io: [("rwasm", Ok(I32(0))), ("wasmtime", Err(StackOverflow))]
  • this branch: results follow in a comment once the local run completes (draft until then).

Impact

Reachable only by modules with tail calls after a non-tail call in the same function. None of the Fluent networks' historical system-runtime hints use return_call, so nothing on chain is affected; the fix is needed before any tail-call payload lands.

Companion: rwasm fix: pin wasmtime-rwasm 45.0.0-rwasm.4 and cover tail calls after a call (test, docs, changelog).

A tail call left the store's rwasm stack counters untouched, on the
assumption that every reader runs right after the call site that wrote
them. That fails for a frame that makes a plain call first: the call
publishes the callee's depth and base, nothing restores them, and the
tail callee's prologue then reads them as its own. A tail-recursive loop
with a call in its body therefore counted one frame per iteration on
Wasmtime and trapped StackOverflow after N_MAX_RECURSION_DEPTH
iterations, while the rwasm VM ran it.

return_call, return_call_indirect and return_call_ref now store the
caller's own counters before the call, like ReturnCallInternal. Modules
without tail calls compile to the same code as before.

Version 45.0.0-rwasm.4. Pinned by rwasm's tests/stack-overflow.rs.
@dmitry123

Copy link
Copy Markdown
Member Author

Local run of rwasm tests/stack-overflow.rs (rwasm v0.7.1 + the test from fluentlabs-xyz/rwasm#224) with the three crates patched to this branch:

  • tail_calls_after_a_call_do_not_count_towards_the_recursion_limit: rwasm.3 → [("rwasm", Ok(I32(0))), ("wasmtime", Err(StackOverflow))]; this branch → both Ok(I32(0)) for return_call and return_call_indirect at 100 × N_MAX_RECURSION_DEPTH iterations.
  • Whole stack-overflow suite: 16 passed. fuel_alignment 43, wide_arithmetic 4, execution_contracts 21, instance_replacement 16, fluentbase 3 passed; lib: test result: ok. 154 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 41.08s ##### DONE (07:59:15)

Ready for review. After merge: tag v45.0.0-rwasm.4 so publish.yml publishes the three crates, then rwasm#224 moves its pin.

tests/all/rwasm_stack.rs drives Config::rwasm_stack_limits directly:
plain recursion traps StackOverflow at max_call_depth (the control),
and a return_call or return_call_indirect made after a plain call in
the same frame runs far past that depth, because the tail call publishes
the caller's own counters. The second case fails on the lowering of
45.0.0-rwasm.3 and passes with the fix.
@dmitry123

Copy link
Copy Markdown
Member Author

Added a fork-local regression test, tests/all/rwasm_stack.rs (cargo test --test all rwasm_stack): it sets Config::rwasm_stack_limits directly, appends the rwasm.frames section itself, and covers two cases.

  • plain_recursion_traps_at_the_depth_limit (control): max_call_depth - 1 nested calls run, one more traps StackOverflow.
  • tail_call_after_a_call_keeps_the_caller_frame: return_call and return_call_indirect after a plain call, 100 × max_call_depth iterations.
lowering control tail call after a call
func_environ.rs from rwasm.3 (d0ea7f7) ok FAILED, StackOverflow
this branch ok ok

@hedwig0x hedwig0x left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the compiler lowering, regression coverage, and crate version updates. The focused rwasm stack tests pass locally; no blocking issues found.

@dmitry123
dmitry123 merged commit 1d1d80b into v45-patched Sep 28, 2026
2 checks passed
@dmitry123
dmitry123 deleted the fix/rwasm-stack-tail-call branch September 28, 2026 03:01
dmitry123 added a commit to fluentlabs-xyz/rwasm that referenced this pull request Sep 28, 2026
The fork release carries the tail-call fix (fluentlabs-xyz/wasmtime#15):
a return_call after a plain call now publishes the caller's rwasm
counters, so the new stack-overflow case passes on both strategies.
dmitry123 added a commit to fluentlabs-xyz/rwasm that referenced this pull request Sep 28, 2026
…all (#224)

* test: pin that a tail call after a call keeps the caller's frame on both backends

A frame that makes a plain call and then a return_call or
return_call_indirect must run at its own depth and base on both
strategies. On wasmtime-rwasm 45.0.0-rwasm.3 the tail callee read the
counters the plain call had published for its callee, so a
tail-recursive loop with a call in its body trapped StackOverflow after
N_MAX_RECURSION_DEPTH iterations on Wasmtime while the rwasm VM ran it.
The case fails on rwasm.3 and passes with the fork fix (rwasm.4).

* docs: correct the tail-call rule of the Wasmtime stack emulation

The pipeline notes claimed a call site restores its own counters after
the call; it does not, the caller keeps them in registers, which is why
a tail call has to publish the caller's counters itself. Record the fix
under Unreleased.

* build(deps): pin wasmtime-rwasm 45.0.0-rwasm.4

The fork release carries the tail-call fix (fluentlabs-xyz/wasmtime#15):
a return_call after a plain call now publishes the caller's rwasm
counters, so the new stack-overflow case passes on both strategies.

* build(deps): pin the fuzz oracle to wasmtime-rwasm 45.0.0-rwasm.4

tests/fuzz_versions.rs requires the differential-fuzz oracle to run the
same wasmtime-rwasm as the root crate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants