Skip to content

Mchp rev2 update - #122

Closed
LianVictor wants to merge 0 commit into
nvidia-holoscan:mainfrom
LianVictor:MCHP-Rev2-update
Closed

Mchp rev2 update#122
LianVictor wants to merge 0 commit into
nvidia-holoscan:mainfrom
LianVictor:MCHP-Rev2-update

Conversation

@LianVictor

@LianVictor LianVictor commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Updated:

  • Configuration and documentation update for Microchip MPF200-ETH-SENSOR-BRIDGE-R2.
  • Updated hsb_flasher protocol for Microchip FPGA's
  • Updated IMX477 python drivers to handle multi-camera input

Additions:

  • Added IMX477 multi camera streaming for RoCE and non-RoCE implementations( up to 4 camera support)
  • Depth Anything V2 implementation for non-RoCE IMX477 as demo. (Instructions can be found within Python file)

Summary by CodeRabbit

  • New Features

    • Added Depth Anything V2 examples with live camera, replay, depth visualization, and interactive display options.
    • Added multi-camera IMX477 player examples supporting up to four cameras and flexible layouts.
    • Added IMX477 variants for PeopleNet, body pose, stereo, and other examples.
    • Added PolarFire R2 firmware and device support.
  • Documentation

    • Updated IMX477 guidance and sensor bridge setup instructions for the MPF200-ETH-SENSOR-BRIDGE-R2.
  • Improvements

    • Improved camera selection and firmware flasher compatibility across supported board versions.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 95e2edf4-2a26-4743-bc8d-1bdde5efd38b

📥 Commits

Reviewing files that changed from the base of the PR and between a2b27cd and 204c1ea.

📒 Files selected for processing (1)
  • examples/depth_anything_v2/depth.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • examples/depth_anything_v2/depth.py

Walkthrough

Changes

Added PolarFire R2 enumeration and firmware flashing support. Added multi-camera IMX477 players and Depth Anything V2 applications. Updated IMX477 example commands and PolarFire R2 hardware documentation.

PolarFire R2 and IMX477 support

Layer / File(s) Summary
PolarFire R2 platform and flashing support
src/hololink/core/*, python/hololink/*, tools/hsb_flasher/...
Added PolarFire R2 identifiers, enumeration configuration, firmware metadata, and version-based flasher selection.
PolarFire R2 hardware setup
docs/user_guide/sensor_bridge_*
Updated device naming, connector details, power requirements, jumper settings, and product references.
Multi-camera IMX477 player pipelines
examples/*multi_imx477_player.py, python/hololink/sensors/imx477.py
Added Linux and socket-based players for up to four IMX477 cameras, with capture, conversion, demosaicing, display, configuration, and cleanup flows.
Depth Anything V2 applications
examples/depth_anything_v2/*
Added replay, V4L2, and IMX477 applications with TensorRT inference, depth postprocessing, interactive display modes, and Holoviz output.
IMX477 example documentation
docs/user_guide/examples.mdx
Added commands for IMX477 player, Tao PeopleNet, body pose, and stereo examples.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

Suggested reviewers: yosilv

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title identifies the Microchip revision 2 update, which matches the main device, firmware, driver, and documentation changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (4)
examples/single_network_multi_imx477_player.py (1)

304-310: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Do not overwrite the core R2 strategy.

The core R2 registration adds board_description and gpio_pin_count. Both players replace that strategy with one that has empty additional metadata. The core registration already uses the required four sensors, two dataplanes, and one SIF per sensor.

  • examples/single_network_multi_imx477_player.py#L304-L310: remove the local set_uuid_strategy call.
  • examples/linux_single_network_multi_imx477_player.py#L287-L293: remove the local set_uuid_strategy call.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/single_network_multi_imx477_player.py` around lines 304 - 310,
Remove the local Enumerator.set_uuid_strategy call and its
BasicEnumerationStrategy setup in
examples/single_network_multi_imx477_player.py:304-310 and
examples/linux_single_network_multi_imx477_player.py:287-293. Preserve the core
R2 registration, including its board_description and gpio_pin_count metadata,
and rely on its existing four-sensor, two-dataplane, one-SIF-per-sensor
configuration.
examples/depth_anything_v2/linux_depth_anything_v2_imx477.py (2)

76-81: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Consider keeping the transpose on the device.

Each frame moves the tensor to host memory and back. On the 4K camera path this adds latency to every frame. cp.ascontiguousarray(cp.moveaxis(tensor, 2, 0)[None]) produces a contiguous device tensor and avoids both copies. Test it before you adopt it, because the comment reports a stride problem with the inference backend.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/depth_anything_v2/linux_depth_anything_v2_imx477.py` around lines 76
- 81, Update the transpose block around the preprocessed tensor to keep the
operation on the device, using CuPy moveaxis and contiguous conversion before
inference. Replace the host round-trip involving tensor.get() and NumPy moveaxis
while preserving the added batch dimension, and validate that the inference
backend no longer encounters the reported stride issue.

335-350: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Release the CUDA primary context in a finally block.

If application.run() raises, the code skips cuDevicePrimaryCtxRelease. Process exit reclaims the context, so the impact is limited to this example script. Nesting the release in its own try/finally makes the cleanup explicit.

♻️ Proposed refactor
     hololink = hololink_channel.hololink()
     hololink.start()
     try:
         hololink.reset()
         camera.configure()
 
         # IMX477 Analog gain settings function. Analog gain value range is 0-1023 in decimal (10 bits). Users are free to experiment with the register values.
         camera.set_analog_gain(0x2FF)
         camera.set_exposure_reg(args.exposure)
 
         application.run()
     finally:
         hololink.stop()
-
-    (cu_result,) = cuda.cuDevicePrimaryCtxRelease(cu_device)
-    assert cu_result == cuda.CUresult.CUDA_SUCCESS
+        (cu_result,) = cuda.cuDevicePrimaryCtxRelease(cu_device)
+        assert cu_result == cuda.CUresult.CUDA_SUCCESS
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/depth_anything_v2/linux_depth_anything_v2_imx477.py` around lines
335 - 350, Move the cuda.cuDevicePrimaryCtxRelease call into a finally block
that wraps the existing hololink startup, camera configuration, and
application.run flow, ensuring primary-context cleanup occurs even when
application.run or earlier setup raises; preserve the existing CUDA success
assertion.
examples/depth_anything_v2/depth.py (1)

151-151: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Derive the split position from the image shape.

self.image_dim hardcodes 518 and duplicates preprocessor.resize_width in depth.yaml. If a user changes the resize width, the interactive split position no longer matches the frame. Use the actual image width instead.

♻️ Proposed refactor
-            pos = int(self.image_dim * self.ratio)
+            pos = int(image.shape[1] * self.ratio)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/depth_anything_v2/depth.py` at line 151, Update the split-position
calculation around self.image_dim in the interactive depth display to derive the
width from the current image/frame shape instead of the hardcoded dimension,
while preserving the existing ratio behavior so changes to the configured resize
width remain aligned with the frame.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/user_guide/sensor_bridge_hardware_setup.mdx`:
- Line 124: Update the Microchip MPF200-ETH-SENSOR-BRIDGE-R2 link in the
hardware setup documentation to target the dedicated R2 product page instead of
the generic Ethernet Sensor Bridge page, keeping the existing link label
unchanged.

In `@examples/depth_anything_v2/depth.py`:
- Around line 38-46: Initialize self.x and self.framebuffer_size with safe
defaults in __init__, alongside the existing display state. Update
toggle_display_mode and cursor_pos_callback to use these initialized values, and
guard any framebuffer-width division so a zero width cannot raise
ZeroDivisionError while preserving normal coordinate behavior.
- Around line 308-311: Update the default config path in the args.config ==
"none" branch to reference the shipped depth.yaml filename instead of
depth_anything_v2.yaml, while preserving the explicit args.config path
unchanged.
- Around line 136-160: Update the depth_colormap handling in both the
“side-by-side” and interactive branches of the display method to convert the
NumPy result from cv2.applyColorMap into a CuPy array before passing it to
cp.hstack. Preserve the existing slicing and concatenation behavior, and apply
the appropriate image format or explicit channel conversion if the displayed
output requires RGB ordering.

In `@examples/single_network_multi_imx477_player.py`:
- Around line 281-286: Restrict the --num-cameras argument to values 1 through 4
by adding choices=range(1, 5) in the argument definitions for
examples/single_network_multi_imx477_player.py:281-286 and
examples/linux_single_network_multi_imx477_player.py:263-268. No other changes
are required.

In `@src/hololink/core/enumerator.cpp`:
- Around line 166-167: Update the branch assigning MICROCHIP_POLARFIRE_R2_UUID
so it uses a distinct R2 discriminator rather than repeating
MICROCHIP_POLARFIRE_BOARD_ID; use the R2-specific board ID or derive the
revision from BOOTP-v1 metadata, ensuring standard boards retain
MICROCHIP_POLARFIRE_UUID and the two-sensor strategy.

---

Nitpick comments:
In `@examples/depth_anything_v2/depth.py`:
- Line 151: Update the split-position calculation around self.image_dim in the
interactive depth display to derive the width from the current image/frame shape
instead of the hardcoded dimension, while preserving the existing ratio behavior
so changes to the configured resize width remain aligned with the frame.

In `@examples/depth_anything_v2/linux_depth_anything_v2_imx477.py`:
- Around line 76-81: Update the transpose block around the preprocessed tensor
to keep the operation on the device, using CuPy moveaxis and contiguous
conversion before inference. Replace the host round-trip involving tensor.get()
and NumPy moveaxis while preserving the added batch dimension, and validate that
the inference backend no longer encounters the reported stride issue.
- Around line 335-350: Move the cuda.cuDevicePrimaryCtxRelease call into a
finally block that wraps the existing hololink startup, camera configuration,
and application.run flow, ensuring primary-context cleanup occurs even when
application.run or earlier setup raises; preserve the existing CUDA success
assertion.

In `@examples/single_network_multi_imx477_player.py`:
- Around line 304-310: Remove the local Enumerator.set_uuid_strategy call and
its BasicEnumerationStrategy setup in
examples/single_network_multi_imx477_player.py:304-310 and
examples/linux_single_network_multi_imx477_player.py:287-293. Preserve the core
R2 registration, including its board_description and gpio_pin_count metadata,
and rely on its existing four-sensor, two-dataplane, one-SIF-per-sensor
configuration.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a4780379-4442-4fd8-9f1b-c24ea894deb6

📥 Commits

Reviewing files that changed from the base of the PR and between 1a1febe and 03a4b10.

⛔ Files ignored due to path filters (2)
  • docs/user_guide/imx477_connect.png is excluded by !**/*.png
  • docs/user_guide/microchip_board.png is excluded by !**/*.png
📒 Files selected for processing (13)
  • docs/user_guide/examples.mdx
  • docs/user_guide/sensor_bridge_firmware_setup.mdx
  • docs/user_guide/sensor_bridge_hardware_setup.mdx
  • examples/depth_anything_v2/depth.py
  • examples/depth_anything_v2/depth.yaml
  • examples/depth_anything_v2/linux_depth_anything_v2_imx477.py
  • examples/linux_single_network_multi_imx477_player.py
  • examples/single_network_multi_imx477_player.py
  • python/hololink/sensors/imx477.py
  • src/hololink/core/enumerator.cpp
  • src/hololink/core/enumerator.hpp
  • tools/hsb_flasher/firmware_flash_strategies/hsb_polarfire.py
  • tools/hsb_flasher/firmware_information/hsb_polarfire_r2.yaml

Comment thread docs/user_guide/sensor_bridge_hardware_setup.mdx Outdated
Comment thread examples/depth_anything_v2/depth.py Outdated
Comment thread examples/depth_anything_v2/depth.py Outdated
Comment thread examples/depth_anything_v2/depth.py Outdated
Comment thread examples/single_network_multi_imx477_player.py Outdated
Comment thread src/hololink/core/enumerator.cpp Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants