feat(presto/slurm): kernel perf flamegraph profiling pipeline

[misiugodfrey]

Summary

Adds a self-contained CPU-time profiling pipeline using Linux kernel perf, with no Perl or CDN dependencies for flamegraph rendering. The pipeline is query-bounded (attach after workers are ready, stop after the query of interest), sharded across TIDs to keep per-worker file sizes manageable, and deferred for symbolization to avoid impact during the benchmark.

Based on kjmph's bench33 POC patch.

New files

• perf_flamegraph.py: dependency-free SVG flamegraph renderer written in Python stdlib; reads perf script text output and writes a single self-contained HTML/SVG file; no FlameGraph.pl, no CDN, no Node.js
• perf_profiler_functions.sh: coordinator-side enable/stop token handshake using filesystem sentinel files; the benchmark loop writes an enable token before each query and a stop token after; the sampler uses these to bound its capture window
• perf-record-sharded.sh: divides the worker's thread list into N shards and runs one perf record process per shard; keeps per-shard perf.data files below a configurable size ceiling
• perf-worker-sampler.sh: host-side orchestration; waits for the enable token, attaches to the worker process, records until the stop token, publishes capture files to the result directory; includes preflight checks for perf_event_paranoid and debug symbol availability
• process-perf-captures.sh: deferred post-processing script run from the login node after the job completes; calls perf script for symbolization and perf_flamegraph.py for rendering; designed to run after job exit so symbolization doesn't interfere with benchmark timing
• presto-repl-from-job.sh: launches an interactive Presto CLI session connected to a running job's coordinator using srun --overlap; useful for inspecting retained query state during a live benchmark

Integration hooks

• functions.sh: run_worker launches perf-worker-sampler.sh as a sidecar srun step when ENABLE_PERF=1
• launch-run.sh: --perf, --perf-output-dir, --perf-frequency flags
• run-presto-benchmarks.sh: ENABLE_PERF propagation and result dir integration

Dependencies

• Depends on PR Land @bdice prototype workflows in main for testing #3 (bench-slurm-robustness) for the result directory structure that perf captures are written into
• Can be reviewed in parallel with PR [DRAFT] CI: Update build and test workflow for Velox #4; no functional dependency between them
• Rebase on PR Land @bdice prototype workflows in main for testing #3 (and optionally PR [DRAFT] CI: Update build and test workflow for Velox #4) after those merge to drop duplicated hunks in the split files

Verification

• Run a GPU benchmark with --perf on the NVL72 cluster; verify perf-worker-sampler.sh attaches to worker processes and produces perf.data.* shard files in result_dir_<jobid>/perf/
• Verify the enable/stop token handshake bounds the capture window to a single query (not the full job duration)
• Run process-perf-captures.sh on the result directory after the job; verify it produces flamegraph.svg (or .html) for each captured worker
• Open the rendered flamegraph in a browser and confirm the SVG is interactive (hover, click-to-zoom)
• Run presto-repl-from-job.sh <job_id> against a live benchmark; verify the Presto CLI connects and can execute queries
• Verify perf-record-sharded.sh splits the TID list correctly and each shard produces a separate perf.data.<n> file
• Run ./ci/check_style.sh on changed Python and shell files

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.