Skip to content

[Nexthop] warm-reboot: disable FLOW_CNT_TRAP before warm shutdown - #4756

Closed
wren-nexthop wants to merge 1 commit into
sonic-net:masterfrom
nexthop-ai:disable-flow-cnt-trap-before-warm-boot
Closed

[Nexthop] warm-reboot: disable FLOW_CNT_TRAP before warm shutdown#4756
wren-nexthop wants to merge 1 commit into
sonic-net:masterfrom
nexthop-ai:disable-flow-cnt-trap-before-warm-boot

Conversation

@wren-nexthop

@wren-nexthop wren-nexthop commented Aug 7, 2026

Copy link
Copy Markdown

What I did

Resolves #4753

Disable the FLOW_CNT_TRAP flex counter group before warm-boot shutdown, and roll the change back if the reboot aborts before kexec.

With trap flow counters enabled, orchagent binds a SAI counter to every HOSTIF trap via SAI_HOSTIF_TRAP_ATTR_COUNTER_ID. After warm boot, syncd replays that attribute during APPLY_VIEW and Broadcom SAI fails with BCM_E_NOT_FOUND, surfacing as SAI_STATUS_ITEM_NOT_FOUND; syncd enters shutdown-wait and orchagent crash-loops. Disabling the group before the warm shutdown lets CoppOrch unbind and clear every trap counter through SAI while the handles are still valid.

How I did it

  • If FLOW_CNT_TRAP is enabled, disable it in CONFIG_DB before the warm shutdown and wait for COUNTERS_TRAP_NAME_MAP to drain.
  • Write a marker file under /host/warmboot (flow_cnt_trap_need_re_enable) so the warmboot-finalizer knows to re-enable the group after a successful reconcile. The marker is only written when re-enable is actually needed, so its absence is the no-op signal.
  • If the warm-reboot aborts before kexec, clear_boot re-enables the counter group and removes the marker.

The finalizer side (re-enable after reconcile) is the companion sonic-buildimage PR: sonic-net/sonic-buildimage#28896

How to verify it

  1. Enable FLOW_CNT_TRAP in CONFIG_DB and config reload.
  2. sudo warm-reboot.
  3. After kexec: links up, no container crash-loops, warmboot finalizer completes.
  4. show flowcnt-trap stats works once the counter group is re-enabled by the finalizer.

Verified on Broadcom Tomahawk-based platforms together with the companion finalizer change; also verified the abort path (interrupted warm-reboot re-enables the group and removes the marker).

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

Signed-off-by: domingo-nexthop <domingo@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warm reboot fails with orchagent crash-loop when FLOW_CNT_TRAP flex counters are enabled

3 participants