[1.7] openhcl_boot: handle multi numa private pool fallback (#3312)#3784
Open
mebersol wants to merge 1 commit into
Open
[1.7] openhcl_boot: handle multi numa private pool fallback (#3312)#3784mebersol wants to merge 1 commit into
mebersol wants to merge 1 commit into
Conversation
On some systems, there is not enough ram in numa node 0 to fufill the private pool allocation request. Change the code to support falling back to allocating the private pool across numa nodes if we cannot match the old behavior. This also opens up the possibility of adding numa awareness to dma_manager and drivers in the future. Note that previous versions of openhcl_boot only support a single range for the private pool, but those systems shouldn't have been able to boot at all, so this change should be safe. Add tests and openvmm test-only capabilities to report multiple numa ranges to the guest.
github-actions
Bot
added
the
release_1.7.2511
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends OpenHCL’s boot-time VTL2 GPA private-pool allocation to work on multi-NUMA configurations by supporting split allocations across NUMA nodes, and threads that capability through OpenVMM/Petri configuration plus new VMM tests.
Changes:
- Add “try NUMA node 0 first, else split across nodes (or force split)” logic for VTL2 GPA pool allocation during OpenHCL boot.
- Add vNUMA-aware guest memory layout construction (per-node sizes) and plumb it through OpenVMM CLI + Petri.
- Add/adjust VMM tests to exercise NUMA-split private pool behavior across boot and servicing boundaries.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| openhcl/openhcl_boot/src/host_params/dt/mod.rs | Implements the NUMA-split fallback logic for VTL2 GPA pool allocation and persists/restores multiple pool ranges. |
| openhcl/openhcl_boot/src/memory.rs | Updates address-space builder to accept multiple persisted pool ranges and adds unit tests for multi-range pool reservation/allocation. |
| openhcl/openhcl_boot/src/cmdline.rs | Adds OPENHCL_VTL2_GPA_POOL_NUMA=split parsing to force split allocation mode. |
| vm/vmcore/vm_topology/src/memory.rs | Adds a NUMA-distributed RAM layout builder plus unit tests for multi-node placement across MMIO gaps. |
| openvmm/openvmm_core/src/worker/dispatch.rs | Switches to NUMA-aware memory layout when numa_mem_sizes is provided, validating totals. |
| openvmm/openvmm_defs/src/config.rs | Extends VM memory config payload with optional per-NUMA-node memory sizes. |
| openvmm/openvmm_entry/src/cli_args.rs | Adds --numa-memory (test-only) CLI to specify per-node RAM sizes. |
| openvmm/openvmm_entry/src/lib.rs | Computes total memory from --numa-memory and populates numa_mem_sizes in config. |
| openvmm/openvmm_entry/src/ttrpc/mod.rs | Initializes the new numa_mem_sizes field in the ttrpc config path. |
| petri/src/vm/mod.rs | Adds numa_mem_sizes to Petri’s memory config (defaulting to None). |
| petri/src/vm/openvmm/construct.rs | Plumbs numa_mem_sizes into OpenVMM config and derives mem_size from the sum when set. |
| vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs | Adds a servicing test that forces split pool allocation on a multi-vNUMA memory config. |
| vmm_tests/vmm_tests/tests/tests/multiarch/memstat.rs | Updates a struct literal to include the new numa_mem_sizes field. |
Comment on lines
+137
to
+149
| // If per-node-size is zero, we're in some strange configuration. We should | ||
| // have been able to allocate this from a single node, as this would mean | ||
| // the number of nodes is larger than the number of pages requested for the | ||
| // pool, so fail explicitly. | ||
| if per_node_size == 0 { | ||
| panic!( | ||
| "cannot split VTL2 pool of size {pool_size_bytes:#x} bytes across \ | ||
| {num_nodes} nodes, per node size {per_node_size:#x} bytes; \ | ||
| enable_vtl2_gpa_pool={enable_vtl2_gpa_pool:?}, \ | ||
| device_dma_page_count={device_dma_page_count:#x?}, \ | ||
| vp_count={vp_count}, mem_size={mem_size:#x}" | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request overview
This PR updates OpenHCL boot-time private pool allocation to support multi-NUMA systems by falling back to allocating the VTL2 GPA pool across NUMA nodes when a single-node allocation (node 0) can’t satisfy the request. It also introduces a command-line option to force NUMA-split behavior and adds VMM tests to exercise the multi-range pool behavior across boot and servicing.
Changes:
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
openhcl/openhcl_boot/src/host_params/dt/mod.rsopenhcl/openhcl_boot/src/memory.rsopenhcl/openhcl_boot/src/cmdline.rsOPENHCL_VTL2_GPA_POOL_NUMA=splitto force NUMA-split pool allocation.vmm_tests/vmm_tests/tests/tests/x86_64/openhcl_uefi.rsvmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs