Skip to content

reboot: use TLS for DPU gNOI calls - #4761

Open
hdwhdw wants to merge 16 commits into
sonic-net:masterfrom
hdwhdw:fix/28540-gnoi-insecure
Open

reboot: use TLS for DPU gNOI calls#4761
hdwhdw wants to merge 16 commits into
sonic-net:masterfrom
hdwhdw:fix/28540-gnoi-insecure

Conversation

@hdwhdw

@hdwhdw hdwhdw commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Why

DPU gNOI servers without provisioned TLS certificates use ephemeral self-signed certificates. SmartSwitch reboot handling must use TLS rather than plaintext and remain robust when the configured legacy port is unavailable.

Tracks sonic-net/sonic-buildimage#28540.

Microsoft ADO (number only): 39179387

What

  • Change DPU gNOI calls from -notls to -insecure.
  • For each DPU, probe the configured port, 8080, and native port 50052 with bounded, side-effect-free System.Time calls; send one bounded Reboot call to the selected port and validate RebootStatus with a bounded call.
  • Propagate individual and parallel DPU failures while preserving whole-switch reboot recovery; restore PCI/transition state on detach or reset failures.
  • Support both modern protobuf descriptors (is_repeated) and older descriptor APIs in DASH dump decoding.
  • Make multi-ASIC pfcstat tests explicitly request all ports when validating frontend and backend output.

Validation

  • pytest -q --noconftest -o addopts='' tests/reboot_smartswitch_helper_test.py - 16 passed
  • A focused dump/dash_util.py protobuf-traversal check passed under protobuf 3.20.3, 4.21.12, and 7.35.1
  • Focused legacy protobuf descriptor test passed under protobuf 7.35.1
  • python3 -m py_compile dump/dash_util.py tests/pfcstat_test.py - passed
  • bash -n scripts/reboot_smartswitch_helper - passed
  • Required Azure run 1188156 passed Static Analysis, Python3, BuildData, and diff coverage
  • One physical DPU0 run using integrated image SONiC.master-28915.1187671-f1a440da3 observed the connection to port 8080 fail, selected port 50052, and completed the exercised gNOI halt/reset workflow

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI lite review requested due to automatic review settings August 7, 2026 22:44
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the SmartSwitch reboot helper to use TLS (with insecure certificate verification) for direct DPU gNOI calls, aligning reboot handling with DPU telemetry servers that only expose ephemeral self-signed TLS.

Changes:

  • Switch DPU gNOI RebootStatus and Reboot invocations from gnoi_client -notls to gnoi_client -insecure.
  • Add pytest coverage that dynamically sources scripts/reboot_smartswitch_helper and asserts TLS flags are used for both command paths.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
scripts/reboot_smartswitch_helper Updates gNOI client invocations to use -insecure (TLS).
tests/reboot_smartswitch_helper_test.py Adds tests that validate the helper issues gNOI commands with TLS enabled.
Suppressed comments (1)

scripts/reboot_smartswitch_helper:192

  • This line uses $() (command substitution), so $? on the following line won’t reliably indicate whether docker exec gnoi_client succeeded; it can also lead to always taking the success path. Run the command directly (no substitution) and quote the -target argument to avoid word-splitting issues.
    $(docker exec gnmi gnoi_client -target ${dpu_ip}:${port} -logtostderr -insecure -module System -rpc Reboot -jsonin '{"method":3, "message":"User initiated reboot"}' &>/dev/null)
    if [ $? -ne 0 ]; then

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/reboot_smartswitch_helper Outdated
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@hdwhdw

hdwhdw commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Physical SmartSwitch validation passed on DPU0 using integrated image SONiC.master-28915.1187671-f1a440da3.

  • The packaged utility observed the connection to port 8080 fail and selected native port 50052.
  • The exercised gNOI halt and hardware-reset workflow completed.
  • Exact baseline images were restored, all four DPUs reported Online, and the testbed was released READY.

Scope boundary: DPU0 ran the RPC acceptance sequence; DPU1-DPU3 were health-checked Online. Host Services was not exercised by this utility validation.

hdwhdw added 2 commits August 8, 2026 22:31
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants