Skip to content

feat(presto/bench): NUMA-aware Presto worker config generation - #391

Draft
misiugodfrey wants to merge 1 commit into
mainfrom
misiug/bench-numa-config
Draft

feat(presto/bench): NUMA-aware Presto worker config generation#391
misiugodfrey wants to merge 1 commit into
mainfrom
misiug/bench-numa-config

Conversation

@misiugodfrey

@misiugodfrey misiugodfrey commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Rewrites generate_presto_config.sh to discover host NUMA topology at config-generation time and size each worker to its proportional share of the NUMA node(s) it will be bound to. Previously all workers on a host received identical configs sized to the full host, which over-allocated CPU and memory for multi-worker-per-host CPU runs.

Based on kjmph's bench33 POC patch.

  • discover_cpu_numa_topology (new helper in common_functions.sh): uses lscpu, lsmem, and /proc/meminfo fallbacks to enumerate CPU NUMA nodes, their logical CPU counts, and their memory in GB; distinguishes CPU-hosting nodes from HBM-only memory nodes (relevant for Grace-Hopper hosts)
  • cpu_numa_node_for_worker (new helper): maps a worker slot index to its NUMA node using balanced contiguous placement (yields 0; 0,1; 0,0,1,1 for 1/2/4 workers on a 2-socket host)
  • count_linux_cpu_list (new helper): parses Linux Cpus_allowed_list-style range strings (e.g. 0-47,96-143) to a count
  • generate_presto_config.sh: NUMA topology discovery runs when USE_NUMA=1; per-worker CPU thread and RAM limits derived from the smallest NUMA node share; CPU_NUMA_WORKER_LAYOUT exported as <node>:<count> map for downstream consumers; set_or_append_property helper for idempotent property file mutations; cuDF exchange guard restricted to GPU-only (was incorrectly gating CPU runs)

Dependencies

Depends on PR #390 (bench-fixes-tuning) for the common_functions.sh jq→python3 fix in wait_for_worker_node_registration. Rebase after PR #390 merges.

Verification

  • On a 2-socket Grace host with CPU_WORKERS_PER_HOST=2 and USE_NUMA=1, verify generate_presto_config.sh produces two configs with CPU count = (total logical CPUs on one socket / 2) and RAM = (HBM on that socket / 2)
  • On a host with USE_NUMA=0, verify the generated config is unchanged from the pre-patch behavior (full-host sizing, single worker)
  • Verify CPU_NUMA_WORKER_LAYOUT is set correctly (e.g. 0:1,1:1 for 2 workers on a 2-socket host)
  • Check that discover_cpu_numa_topology correctly ignores HBM-only NUMA nodes as CPU-hosting nodes on Grace-Hopper (nodes with CPU_NUMA_NODE_CPU_COUNTS[i] == 0 should not contribute CPU slots)
  • Verify set_or_append_property is idempotent: running generate_presto_config.sh twice does not duplicate property entries
  • Run ./ci/check_style.sh on changed files

@copy-pr-bot

copy-pr-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Based on kjmph's bench33 POC patch.

Rewrites generate_presto_config.sh to discover host NUMA topology and
size each worker to its share of the NUMA node(s) it's bound to,
rather than dividing the full host uniformly.

- discover_cpu_numa_topology / cpu_numa_node_for_worker helpers in
  common_functions.sh using lscpu, lsmem, and /proc/meminfo fallbacks
- CPU_WORKERS_PER_HOST controls how many workers to spread per host
- Per-worker CPU thread and RAM limits derived from the smallest NUMA
  node share assigned anywhere on the host (homogeneous sizing)
- CPU_NUMA_WORKER_LAYOUT exported for downstream consumers
- set_or_append_property helper for idempotent config file mutations
- cuDF exchange guard restricted to GPU-only (was incorrectly gating
  CPU configurations)
@misiugodfrey
misiugodfrey force-pushed the misiug/bench-numa-config branch from 5eb4e2e to dcfa364 Compare July 28, 2026 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant