Repository navigation
Fix scan platform probes: own-cgroup memory limit, OMP_PLACES binding, scan file growth on rerun - #817
Open
jtravs wants to merge 3 commits into
Open
Fix scan platform probes: own-cgroup memory limit, OMP_PLACES binding, scan file growth on rerun#817jtravs wants to merge 3 commits into
jtravs wants to merge 3 commits into
Conversation
The scan admission read only the root cgroup's memory.max (and the v1 equivalent), so under a cgroup manager without Slurm variables the workers were admitted against the whole machine. cgroup_memory in solprop._process now follows /proc/self/cgroup up to the mount, as the CPU quota reader does, taking the tightest memory.max or memory.limit_in_bytes. An unreadable or malformed limit file and a malformed SOLPROP_SCAN_CPU_OFFSET or SOLPROP_SCAN_CPU_COUNT now raise DriverError naming the file or variable. Closes #170 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPwwkUUG9KSAwneHjgH
libgomp binds the main thread to its first place when OMP_PLACES is set and OMP_PROC_BIND is unset (measured with OMP_PLACES=cores: one core's two hardware threads). allowed_cpus checked only OMP_PROC_BIND, so that binding was believed as the grant and every pool sized to one core. OMP_PROC_BIND=false still turns binding off. Closes #172 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPwwkUUG9KSAwneHjgH
Every close deleted and rebuilt the consolidated layouts, and HDF5 does not reclaim a deleted group's space, so an unchanged rerun grew the file. A commit now marks the layouts stale durably before it writes, and close rebuilds only stale layouts, so a run killed before consolidating is still rebuilt by the next close. The guide states what a rerun that commits points costs. Closes #169 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPwwkUUG9KSAwneHjgH
jtravs
marked this pull request as ready for review
November 13, 2026 19:29
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Documentation build overview
25 files changed ·
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SolpropError#170: scan admission reads the tightest memory limit on the process's own cgroup path (v2memory.maxand v1memory.limit_in_bytes, walking up to the mount) instead of the root group's. A malformed limit file or a malformedSOLPROP_SCAN_CPU_OFFSET/COUNTraisesDriverErrornaming it. The path walk is shared with the existing CPU-quota reader. Tests use fake cgroup trees.OMP_PLACESwithoutOMP_PROC_BINDcaps every thread pool at the bound core #172:allowed_cpustreats binding as on whenOMP_PROC_BINDis anything butfalse, or when it is unset andOMP_PLACESis set, as libgomp does. Checked on Linux with libgomp:OMP_PLACES=coresalone bound the main thread to one core. libomp was not checked.layouts. A durablecurrentflag is cleared before a commit and set after the rebuild is flushed, so an interrupted close still rebuilds next time. No-op reruns now grow the file only by the execution row (~6 kB here, HDF5's per-dataset cost). A rerun that commits points still copieslayouts; that cost is documented in the guide, and a follow-up is proposed separately.Closes #170
Closes #172
Closes #169
🤖 Generated with Claude Code
https://claude.ai/code/session_01XGENPwwkUUG9KSAwneHjgH