Replace libunwind with framehop based library#62395
Draft
gbaraldi wants to merge 2 commits into
Draft
Conversation
…P=1) Adds an async-signal-safe, framehop-backed unwinder for the *backtrace* path (stacktraces, exceptions, the profiler signal handler), replacing the local-unwinding use of libunwind that can deadlock in signal handlers (dl_iterate_phdr / malloc under the loader lock) and is ~3-4x slower. The unwinder lives in an external crate (gbaraldi/framehopunwind), linked into libjulia-internal (stackwalk.c) and libjulia-codegen (debuginfo.cpp register_eh_frames), which share its single loaded instance. It is installed from the prebuilt LibFramehopUnwind_jll v0.1.2 artifact by default (bb-install; built in Yggdrasil from the pinned framehopunwind commit), with a cargo source build behind USE_BINARYBUILDER_FRAMEHOPUNWIND=0 (git-external, like mmtk_julia). Opt-in via USE_FRAMEHOP=1; JL_USE_FRAMEHOP is derived for Linux/FreeBSD/macOS on x86_64/aarch64. When off, the build is byte-for-byte unchanged. libunwind stays linked (task.c context switching + proc-info lookup still use it). On macOS bt_context_t is a mach thread state, so registers are extracted via fh_context_from_thread_state, and the forceDwarf retry is compiled out. Verified on Linux x86_64 (both the jll artifact and the source build): Julia's backtrace, stacktraces, and Profile suites pass, backtraces/ exceptions/profiler match libunwind frame-for-frame, framehop is the live unwinder (no libunwind stepping symbols imported), and backtrace capture is ~3-4x faster. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flip the default on so every CI pipeline builds and tests with the framehop unwinder. Guarded to the supported set (Linux/FreeBSD/Darwin on x86_64/aarch64, libunwind enabled, non-musl — LibFramehopUnwind_jll has no musl artifact yet) so other pipelines build unchanged; an explicit USE_FRAMEHOP=0 still overrides (use for sanitizer pipelines if the uninstrumented unwinder library proves noisy under TSan/ASan). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Libunwind has been very unreliable. Specially on aarch64 platforms, being slow and dropping frames. https://github.com/mstange/framehop is a rust based unwinding library that seems better supported. I tasked the robots to build https://github.com/gbaraldi/framehopunwind. Which exposes a libunwind like C API for us to use. https://github.com/gbaraldi/framehopunwind
This is much faster and more reliable on aarch64 apple on my testing but it needs further checking