clangsa: add thread-safety capability analysis of safepoint annotations#62288
Conversation
e733f0a to
0090f6e
Compare
fingolfin
left a comment
There was a problem hiding this comment.
Thanks for working on this! An obvious blocker is that doc/src/devdocs/gc-sa.md is not being updated to document the new annotation.
Do I understand right that the plan is that instead of annotating functions that do not contain a safepoint, we instead annotation function that might contain a safepoint? So we kinda flip what the "default" assumption is.
0090f6e to
5aea342
Compare
|
That's right. We need about half as many annotations for CANSAFEPOINT than we had for NOTSAFEPOINT, and we're now able to correctly annotate more of codegen. It also means we can remove the list of all of the exceptions to that, and exceptions to the exceptions, and the exceptions to those, and so on (though CANSAFEPOINT does have some exceptions too). Because it can be checked with a total proof, it also means we validate it separately from gc-analyzer, making the heuristic gc algorithm more precise (by removing unnecessary states). |
f5faf0d to
112bcd3
Compare
fingolfin
left a comment
There was a problem hiding this comment.
I'll try to look through this, but it's a lot. For now a quick comment on the doc changes (thanks for those!)
112bcd3 to
408ef75
Compare
Documentation updated. Will address other feedback post-merge as well–there's many other places to fix, this is just trying to get in the framework so that we can enforce LLM stops introducing more mistakes.
Add an Clang Thread Safety Analysis ("capability") model of Julia's
safepoints, selected by compiling with `make safesrc` parallel to how
-D__clang_gcanalyzer__ enables the GCChecker annotations. This turns the
unreliable NOTSAFEPOINT thread-safety checks into mandatory-enforced
capabilities.
Implementation: the safepoint annotations map onto a zero-sized
jl_can_safepoint token capability: JL_CANSAFEPOINT requires it,
JL_NOTSAFEPOINT_ENTER and JL_NOTSAFEPOINT_LEAVE release and acquire it,
and JL_NOTSAFEPOINT holds nothing, so -Wthread-safety flags a
JL_NOTSAFEPOINT function that reaches a safepoint. Because this checker
has no implicit default, it requires JL_CANSAFEPOINT on every function
that is not JL_NOTSAFEPOINT.
To add those annotations in bulk, there was a
julia-safepoint-annotations clang-tidy check auto fixit script used,
which inserted JL_CANSAFEPOINT on the first declaration of any function
that lacks a safepoint dispositio. This then required many manual fixes
afterwards, since there were many mistakes in the NOTSAFEPOINT
annotations, a few places where the old default-safe analysis wasn't
permitted under the new stricter analysis. There are still a few places
where the analyzer is disabled due to existing bugs.
Also teach the GCChecker to treat any JL_CANSAFEPOINT function as a
possible safepoint even when it is inlined and no safepoint is found in
its body, so the annotation is honored as a promise rather than
re-derived from the body. This is a prelude to erasing JL_NOTSAFPOINT
entirely, since it is no longer necessary.
Lastly, we teach our clang-tidy first-decl pass to ensure this
capability is tracked through function pointers also (upstream LLVM
cannot do this).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nnotations Simplifying the logical flow of jl_record_backtrace also revealed some latent suboptimal behavior, which we can address, though is not functionally relevant normally.
408ef75 to
58adf05
Compare
Add an Clang Thread Safety Analysis ("capability") model of Julia's safepoints, selected by compiling with
make safesrcparallel to how -D__clang_gcanalyzer__ enables the GCChecker annotations. This turns the unreliable NOTSAFEPOINT thread-safety checks into mandatory-enforced capabilities CANSAFEPOINT. This is much faster, more reliable, and the error messages are more precise.tl;dr; in the future, annotate functions with CANSAFEPOINT family macros if they can reach a safepoint (e.g. an allocation function). In a subsequent PR, I'll remove all NOTSAFEPOINT annotations.
Implementation: the safepoint annotations map onto a zero-sized jl_can_safepoint token capability: JL_CANSAFEPOINT requires it, JL_NOTSAFEPOINT_ENTER and JL_NOTSAFEPOINT_LEAVE release and acquire it, and JL_NOTSAFEPOINT holds nothing, so -Wthread-safety flags a JL_NOTSAFEPOINT function that reaches a safepoint. Because this checker has no implicit default, it requires JL_CANSAFEPOINT on every function that is not JL_NOTSAFEPOINT.
To add those annotations in bulk, there was a
julia-safepoint-annotations clang-tidy check auto fixit script used, which inserted JL_CANSAFEPOINT on the first declaration of any function that lacks a safepoint dispositiom. This then required many manual fixes afterwards, since there were many mistakes in the NOTSAFEPOINT annotations, a few places where the old default-safe analysis wasn't permitted under the new stricter analysis. There are still a few places where the analyzer is disabled due to existing bugs.
Also teach the GCChecker to treat any JL_CANSAFEPOINT function as a possible safepoint even when it is inlined and no safepoint is found in its body, so the annotation is honored as a promise rather than re-derived from the body. This is a prelude to erasing JL_NOTSAFPOINT entirely, since it is no longer necessary.
Lastly, we teach our clang-tidy first-decl pass to ensure this capability is tracked through function pointers also (upstream LLVM cannot do this). We'll unfortunately also eventually need to make some bugfixes to the clang-23 that is currently pre-release to handle function pointers correctly.