Repository navigation
Test that libCling keeps LLVM private - #23606
Open
vgvassilev wants to merge 6 commits into
Open
vgvassilev wants to merge 6 commits into
vgvassilev wants to merge 6 commits into
Conversation
Test Results 24 files 24 suites 4d 1h 6m 59s ⏱️ For more details on these failures, see this check. Results for commit e5fe103. ♻️ This comment has been updated with latest results. |
libCling links LLVM and Clang statically and hides them with -Bsymbolic and libCling.script, so that another LLVM in the process, such as Triton's (PyTorch's compiler), neither binds to libCling's nor has libCling bind to its own. Nothing tests this. When the hiding was incomplete, ALICE saw sporadic crashes with a second libLLVM.so in the process (root-project#19889), and linking libCling against a shared libLLVM, which cannot be hidden, would bring them back unnoticed. Run root.exe and rootcling with a preloaded canary that defines LLVM functions libCling calls, and fail if one of them is reached; a self-test checks that the canary catches a library that relies on the LLVM of its host. With glibc, also fail if the dynamic linker's own log, LD_DEBUG=bindings, shows a binding of an LLVM or Clang symbol across libCling's boundary. Check that the canary's names exist in the LLVM ROOT was built with, so that the test cannot go stale. The tests are ELF only, and skipped for AddressSanitizer builds, which control LD_PRELOAD themselves. Related: the 'Option registered more than once' error with an external LLVM (root-project#12156), and the discussions in root-project#20903 and root-project#23546 about rootcling's llvm::cl options and linking against a shared libLLVM.
Two-level namespaces keep libCling's imports on macOS private, so a canary cannot reach them as it does on Linux. What another LLVM in the process can reach are libCling's exports: dyld merges weak definitions across images by name, and lookups by name, as for Python extensions linked with -undefined dynamic_lookup or through dlsym(), see every export. Audit libCling's loaded image: it must use two-level namespaces, load no shared LLVM or Clang library, and export no LLVM or Clang symbols. A self-test runs the audit on a library that exports two, one of them weak, so that a broken audit cannot pass. The test fails today: libCling exports 23,030 such symbols, 120 of them weak (arm64, minimal build).
On Windows, imports are bound to the DLL that provides them, so another LLVM cannot interpose on libCling's as it can on Linux. Two LLVMs mix when libCling imports a shared LLVM or Clang DLL instead of linking LLVM statically: once a DLL of a given name is loaded, the loader uses it for every module that imports a DLL of that name, wherever it was found [1]. Check that libCling.dll imports no LLVM or Clang DLL, also with delayed loading, reading its import tables from the file. A self-test runs the check on a library importing from a stand-in "LLVM" DLL, so that a broken check cannot pass. The test logs the DLLs it finds, for diagnosing CI. libCling's exports are not checked: only lookups by name across all modules, as some JITs do, reach them. [1] https://learn.microsoft.com/windows/win32/dlls/dynamic-link-library-search-order
vgvassilev
force-pushed
the
metacling-test-llvm-isolation
branch
from
October 4, 2026 14:35
bf29de3 to
2a19f2d
Compare
libCling.script hides the LLVM and Clang symbols that -fvisibility=hidden leaves visible, mostly template instantiations, but only on Linux: ld64 takes no version script. On macOS, libCling exports 23,030 of them, 120 as weak definitions. Two-level namespaces protect libCling's own imports, but dyld merges weak definitions across images by name, and lookups by name, as with -undefined dynamic_lookup or dlsym(), see every export: the macOS form of root-project#19889. Generate ld64's exported symbols list from libCling.script, which stays the one list of what libCling exports, and fail the configuration if it yields none. test-metacling-llvm-isolation-exports now passes. In a minimal build, no other ROOT library imports LLVM or Clang symbols from libCling.
With LLVM assertions, on x86_64 macOS, the interpreter aborts right after building the module for <mach-o/dyld.h> that the macro of test-metacling-llvm-isolation-exports includes: Assertion failed: (!FoundVal && "Key already in new map?"), function moveFrom, file DenseMap.h, line 448. Define the few Mach-O structures the audit reads, whose layout the ABI fixes, and get dyld's image functions through dlsym(), so that the interpreter needs no Mach-O header.
Restore the line breaks of the CALLER role's description, and document the PROVIDER and LEAKY roles that the macOS and Windows tests added.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After a discussion with @guitargeek, I realized that some of the expressed concerns can be demonstrated with not a lot of non-trivial code. This pull request tries to establish a ground truth and demonstrate the issues arising from not hiding the llvm symbols. If/When that PR lands I can provide a fix that will also unblock the
-Dbuiltin_llvm=Offwork and will provide sufficient runtime guarantees and reporting if a clash happens.libCling links LLVM and Clang statically and hides them with -Bsymbolic and libCling.script, so that another LLVM in the process, such as Triton's or a graphics driver's, neither binds to libCling's nor has libCling bind to its own. Nothing tests this. When the hiding was incomplete, ALICE saw sporadic crashes with a second libLLVM.so in the process (#19889), and linking libCling against a shared libLLVM, which cannot be hidden, would bring them back unnoticed.
Run root.exe and rootcling with a preloaded canary that defines LLVM functions libCling calls, and fail if one of them is reached. With glibc, also fail if LD_DEBUG=bindings logs a binding of an LLVM or Clang symbol across libCling's boundary; this relies only on the dynamic linker. Check that the canary's names exist in the LLVM ROOT was built with, so that the test cannot go stale.
Related: the 'Option registered more than once' error with an external LLVM (#12156), and the discussions in #20903 and #23546 about rootcling's llvm::cl options and linking against a shared libLLVM.