Skip to content

feat: Basic-block & Register tracing & Trampoline update - #571

Open
4ar0nma0 wants to merge 3 commits into
eunomia-bpf:masterfrom
4ar0nma0:master
Open

feat: Basic-block & Register tracing & Trampoline update#571
4ar0nma0 wants to merge 3 commits into
eunomia-bpf:masterfrom
4ar0nma0:master

Conversation

@4ar0nma0

Copy link
Copy Markdown

This pull request introduces significant improvements to the PTX attach and trampoline infrastructure, focusing on enhanced support for basic block (BB) tracepoints, improved PTX version compatibility, and better CUDA kernel function handling. The changes include a new pass for BB tracepoints, more robust fallback mechanisms for patched kernel launches, and a smarter PTX compiler that can automatically retry with a compatible version. Several device helper functions and build scripts have also been updated for modern CUDA support and reliability.

Key changes:

BB Tracepoint Support

  • Added a new shared library pass, ptxpass_bb_tracepoint, with its own CMake configuration and integration into the build system, enabling BB tracepoint instrumentation. (attach/nv_attach_impl/pass/ptxpass_bb_tracepoint/CMakeLists.txt, attach/nv_attach_impl/CMakeLists.txt) [1] [2]
  • Updated the default configuration of the kprobe entry pass to exclude BB tracepoints, ensuring proper separation of instrumentation logic. (attach/nv_attach_impl/pass/ptxpass_kprobe_entry/main.cpp)

CUDA Kernel Function Management

  • Added find_single_patched_kernel_function() to allow fallback to a single available patched kernel when multiple mappings are unavailable, improving late attach reliability. This fallback is now used in both CUDA runtime and driver API launch paths. (nv_attach_impl.cpp, nv_attach_impl.hpp, nv_attach_impl_frida_setup.cpp) [1] [2] [3] [4]
  • Enhanced kernel function prefill logic to heuristically map BB tracepoint and base kernel names, increasing the chances of successful patching after loading fatbins. (nv_attach_impl.cpp) [1] [2]

PTX Compiler Robustness

  • Improved the PTX compiler to automatically detect and retry compilation with the PTX version suggested by compiler diagnostics, increasing compatibility with different CUDA toolchains. (ptx_compiler.cpp) [1] [2] [3]
  • Added helper functions for rewriting PTX version strings and extracting supported versions from error logs. (ptx_compiler.cpp)

Device Helper and Build System Updates

  • Registered new external helper functions (get_ptx_reg, get_reg_count) and updated device-side trampoline code for better argument preservation and atomic operations. (ptxpass_core/src/core.cpp, trampoline/default_trampoline.cu) [1] [2] [3] [4]
  • Modernized the trampoline build system: switched to nvcc for PTX generation, updated the default architecture to sm_75, and improved the documentation accordingly. (trampoline/Makefile, trampoline/README.md) [1] [2]

Miscellaneous

  • Added missing includes for <algorithm> and <cctype> to ensure portability and correctness. (nv_attach_impl.cpp, ptx_compiler.cpp) [1] [2]

These changes collectively improve instrumentation flexibility, reliability of kernel patching, and compatibility with modern CUDA environments.

void nv_attach_impl::register_custom_helpers(
ebpf_helper_register_callback register_callback)
{
register_callback(501, "bpf_puts", (void *)bpftime_nv_helper_stub);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registering here is not necessary. These functions are already registered at

vm.register_external_function(1, "map_lookup", (void *)test_func);

@Officeyutong

Copy link
Copy Markdown
Contributor
  • There should be some unit tests for the basic block parser. We may use Claude or Codex to generate unit tests and tell it try the best to find a pattern that makes the parser failed to parse.
  • CI seems to be broken

@4ar0nma0

Copy link
Copy Markdown
Author

CI fix. Need to check if it is a CI config problem. Parser verification will be after that

@yunwei37

Copy link
Copy Markdown
Member

CI triage update (2026-07-16): this PR has real GPU regressions, not just runner queueing. The failing checks show two distinct issues: (1) directly_run_on_gpu reaches the direct nvPTXCompiler path and does not apply the PTX version downgrade/retry used elsewhere, so generated PTX with .version 9.2 is rejected by a compiler supporting up to 8.5; (2) cuda-counter builds a trampoline from nvcc-generated PTX and then collides on duplicate $str symbols between app and trampoline PTX. This branch is also behind/conflicting with current master, so a safe fix should start with a rebase/conflict resolution and then keep the patch scoped to PTX version normalization plus trampoline symbol hygiene. I did not force-push into the contributor fork.

@yunwei37

Copy link
Copy Markdown
Member

CI triage update:

I found two real CUDA/PTX regressions in the available logs, separate from the current self-hosted runner queue issue:

  • cuda-counter path: the PR changes the trampoline generated by nvcc. The generated PTX advertises .version 9.2, then the runtime downgrade/retry path rewrites it toward 8.5. After that, compilation hits duplicate PTX symbols such as $str between the application module and the trampoline module.
  • directly_run_on_gpu path: the direct nvPTXCompiler path does not appear to use the same PTX-version downgrade/retry behavior, so it fails directly on unsupported PTX ISA 9.2 vs the installed compiler support for 8.5.

This PR is also behind master and has conflicts, and its head branch is an external fork default branch. I did not push a fix because rewriting this branch from the maintainer side would be risky.

Suggested fix direction after rebase/conflict resolution:

  • generate trampoline PTX with unique/internalized string/symbol names, or link it in a way that cannot collide with user PTX symbols;
  • centralize the PTX version compatibility retry so the direct nvPTXCompiler path and cuda-counter path behave consistently;
  • add a minimal regression test that compiles app PTX plus trampoline PTX containing duplicate-looking string labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants