[rocm, telemetry] fix: map visible ranks for SMI utilization - #88
Conversation
|
Warning Review limit reached
More reviews will be available in 41 minutes and 2 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds ChangesROCm Visible-Rank Telemetry Mapping
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a robust ROCm visible-rank telemetry resolution mechanism. It adds a new utility module rocm_visibility.py to parse and resolve ROCm visibility environment variables (ROCR_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES, and CUDA_VISIBLE_DEVICES), mapping visible device ranks to physical ROCm SMI indexes. This ensures that keepalive backoff and GPU listing telemetry align correctly with the visible ordinals passed by users. Comprehensive mock-based unit tests have been added to verify various environment configurations, and documentation across the repository has been updated to reflect these changes. I have no feedback to provide as there are no review comments.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
tests/utilities/test_gpu_info.py (1)
361-365: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAssert
physical_idis omitted when unresolved.
gpu_info._query_rocmonly addsphysical_idwhen known, butinfo.get("physical_id")would also pass if the API started returning"physical_id": None.Proposed stronger assertion
- assert [info.get("physical_id") for info in infos] == [None, None] + assert all("physical_id" not in info for info in infos)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/utilities/test_gpu_info.py` around lines 361 - 365, The ROCm GPU info test is too weak because `info.get("physical_id")` passes even if `physical_id` is present with a null value. Update the assertions in `test_gpu_info` for `gpu_info._query_rocm` to verify that each entry does not contain the `physical_id` key at all when it cannot be resolved, while keeping the existing checks for `id`, `visible_id`, and `utilization`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/keep_gpu/utilities/gpu_info.py`:
- Around line 177-180: The busy-percent probe in gpu_info.py’s util query
currently catches Exception, which triggers Ruff BLE001; narrow the except in
the rocm_smi.rsmi_dev_busy_percent_get path to the specific expected failure
type if possible, or add an explicit inline suppression with a brief best-effort
telemetry rationale while keeping the logger.debug fallback behavior intact.
In `@src/keep_gpu/utilities/rocm_visibility.py`:
- Around line 28-31: The broad except in the hardware probe inside the count
parsing logic triggers Ruff BLE001; update the exception handling in the count
retrieval path to either catch only the expected failure types around count_fn()
/ int(...) or add a targeted noqa: BLE001 with a brief intent comment if this
fallback-to-None behavior is intentional. Use the existing count_fn-based probe
in the visibility utility to keep the change localized.
---
Nitpick comments:
In `@tests/utilities/test_gpu_info.py`:
- Around line 361-365: The ROCm GPU info test is too weak because
`info.get("physical_id")` passes even if `physical_id` is present with a null
value. Update the assertions in `test_gpu_info` for `gpu_info._query_rocm` to
verify that each entry does not contain the `physical_id` key at all when it
cannot be resolved, while keeping the existing checks for `id`, `visible_id`,
and `utilization`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 959930da-6f16-439b-ae05-1ca103bcb991
📒 Files selected for processing (15)
AGENTS.mdREADME.mddocs/concepts/architecture.mddocs/getting-started.mddocs/guides/cli.mddocs/guides/mcp.mddocs/guides/python.mddocs/plans/rocm-visible-rank-telemetry.mddocs/reference/api.mddocs/reference/cli.mdsrc/keep_gpu/single_gpu_controller/rocm_gpu_controller.pysrc/keep_gpu/utilities/gpu_info.pysrc/keep_gpu/utilities/rocm_visibility.pytests/rocm_controller/test_rocm_utilization.pytests/utilities/test_gpu_info.py
1 participant
Summary
ROCR_VISIBLE_DEVICESplus one matching HIP/CUDA overlay before querying ROCm SMI utilization.id/visible_idas start-compatible visible ordinals and addphysical_idonly when the ROCm SMI index is known.Test Plan
physical_id.PYTHONPATH=$PWD/src pytest tests/rocm_controller tests/utilities/test_gpu_info.py -q(26 passed, 1 skipped)PYTHONPATH=$PWD/src pytest tests -q(235 passed, 11 skipped)PYTHONPATH=$PWD/src mkdocs build(passed with existing Material warning and unlisted plan-page notices)pre-commit run --all-filesgit diff --check && git diff --cached --checkSummary by CodeRabbit
New Features
Bug Fixes
Documentation