Skip to content

feat: add GPU SIMT-aware eBPF verifier - #561

Open
yunwei37 wants to merge 2 commits into
masterfrom
feat/gpu-verifier
Open

feat: add GPU SIMT-aware eBPF verifier#561
yunwei37 wants to merge 2 commits into
masterfrom
feat/gpu-verifier

Conversation

@yunwei37

Copy link
Copy Markdown
Member

Summary

  • Add a GPU-specific SIMT-aware verification pass for device-side eBPF programs
  • Runs PREVAIL (standard eBPF safety) + SIMT-specific checks (uniformity, divergence, atomics, budgets)
  • Integrated into nv_attach_impl GPU loading path, called before PTX compilation
  • 24 test cases, 175 assertions, all passing

Verifier Pipeline

eBPF bytecode → PREVAIL (memory safety, loops, types)
              → Uniformity analysis (CFG dataflow, stack tracking, helper out-params)
              → SIMT safety (varying branches, prohibited helpers, atomic addresses, map keys)
              → Resource budget (instruction/helper/memory limits with loop scaling)
              → PASS → PTX compilation

Key Design Decisions

  • PREVAIL unmodified: GPU helpers registered via set_non_kernel_helpers(), no submodule changes
  • Metadata-driven: helper uniformity/effects from gpu_platform table, not hard-coded IDs
  • Stack-sensitive: tracks uniformity per stack byte; helper pointer out-params propagate to stack
  • Map-value provenance: loads from map_lookup results inherit key uniformity

Eval Results

Verification adds ~3ms to policy load (<1% of CUDA attach overhead). SIMT pass itself is 18-1284 μs for 16-2048 instruction programs.

  • Correctness: 10/10 expected verdicts (unsafe rejected, safe accepted)
  • False positives: 0/18 GPU example programs falsely rejected
  • 5 GPU-specific hazard classes caught that standard eBPF verification misses:
    lane-varying branches, prohibited helpers, non-uniform atomics, varying map keys, resource budget violations
  • Live GPU demo (RTX 5090): safe program loads and runs correctly in STRICT mode; unsafe program rejected at load time

Suggested eval paragraph for paper (RQ3)

The SIMT-aware verifier is a load-time mechanism: it adds approximately 3 ms per device-side policy, less than 1% of the total CUDA attach overhead (which is dominated by PTX compilation at $\sim$450 ms). All device-side policies and observability tools presented in this paper pass verification without modification. The verifier rejects programs that violate GPU-specific safety invariants---lane-varying branch conditions, non-uniform atomic addresses, prohibited synchronization primitives, lane-varying map-update keys, and per-hook resource budget violations---none of which are detected by standard eBPF verification. For example, a policy that conditionally branches on \texttt{threadIdx} (causing warp divergence) is rejected at load time with a diagnostic identifying the non-uniform predicate.

Test plan

  • bpftime_gpu_verifier_tests — 35 assertions, 4 test cases
  • bpftime_gpu_verifier_e2e_tests — 30 assertions, 4 test cases (compiled .bpf.c → bytecode → verifier)
  • bpftime_gpu_verifier_stress_tests — 99 assertions, 10 test cases (gap-specific coverage)
  • bpftime_verifier_tests — 11 assertions, 6 test cases (CPU regression)
  • Live GPU demo on RTX 5090

🤖 Generated with Claude Code

Add a GPU-specific verification pass that extends standard eBPF safety
checks with SIMT-aware constraints for safe device-side policy execution.

The verifier pipeline runs PREVAIL (standard eBPF memory safety, bounded
loops, type checking) followed by GPU-specific SIMT checks:

- Warp-uniformity dataflow analysis: CFG-based fixed-point iteration
  tracking register and stack-slot uniformity, with metadata-driven
  helper classification (uniform vs lane-varying return values and
  out-parameter effects)
- SIMT safety checks: reject lane-varying branch conditions,
  prohibited helpers (e.g., membar), non-uniform atomic addresses,
  and lane-varying map-update keys (checked via stack-byte uniformity
  with map-descriptor-aware key sizes)
- Per-hook resource budgets with PREVAIL stats-based loop scaling
- Map-value pointer provenance: loads from map_lookup results inherit
  key uniformity; context loads remain uniform (warp-leader model)
- Warp-aggregation helper infrastructure for future __ballot_sync etc.

Integration:
- Verifier called in nv_attach_impl before PTX compilation
- Supports BPFTIME_VERIFIER_LEVEL: STRICT (reject) / WARNING / NO_VERIFY
- Strict rejection cleanly aborts the GPU attach process
- GPU helpers registered via set_non_kernel_helpers() so PREVAIL
  accepts them without submodule modifications

Test coverage: 24 test cases, 175 assertions
- Unit tests: uniformity analysis, SIMT safety, resource budget
- E2E tests: compiled .bpf.c programs (unsafe rejected, safe accepted)
- Stress tests: stack propagation, multi-byte keys, dead code, unknown
  helpers, helper out-params, loop-aware budgets, descriptor fallback

Eval data (benchmark/gpu/verifier/):
- Correctness: 10/10 expected verdicts matched
- False positives: 0/18 example programs falsely rejected
- SIMT pass performance: 18-1284 μs (16-2048 instructions)
- Live GPU demo (RTX 5090): verification adds ~3ms to policy load
  (<1% of total CUDA attach overhead)
- Comparison: catches 5 GPU-specific hazard classes that standard
  eBPF verification misses

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pass map descriptors explicitly to analyze_uniformity() instead of
  reading global state (critical correctness fix)
- Remove accidentally committed 1.5MB ELF binary from live_demo
- Fix -Wmissing-field-initializers warnings (add dump_btf_types_json)
- Update safety_demo_results.md to reflect fail-closed fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@yunwei37

Copy link
Copy Markdown
Member Author

CI triage update (2026-07-16): this branch is substantially behind/conflicting and has broad old failures/cancellations across CPU, runtime, Docker, bpftrace, and GPU jobs. This is not a narrowly fixable CI flake from current logs. It needs a rebase onto current master and then a fresh failure pass before making targeted fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant