Skip to content

[dhcpmon] Match DHCPv4 receive and transmit counters across windows - #103

Closed
Xichen96 wants to merge 21 commits into
sonic-net:masterfrom
Xichen96:dev/xichenlin/track-unmatched-v4-windows
Closed

[dhcpmon] Match DHCPv4 receive and transmit counters across windows#103
Xichen96 wants to merge 21 commits into
sonic-net:masterfrom
Xichen96:dev/xichenlin/track-unmatched-v4-windows

Conversation

@Xichen96

@Xichen96 Xichen96 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Description of PR

Track unmatched DHCPv4 RX activity per message type with absolute counter
watermarks and adjacent-window TX credit.

Work item tracking
  • Microsoft ADO (number only): 38615655

Type of change

  • Bug fix

Approach

  • Maintain RX/TX watermarks independently for Discover, Offer, Request, and
    Ack.
  • Preserve an unmatched RX across idle windows.
  • Clear it when matching TX activity appears in the current or credited
    adjacent window.
  • Preserve one-window TX-before-RX credit.
  • Seed watermarks before the first monitoring window.
  • Reset safely when counters decrease or are cleared.
  • Keep an idle flow INDETERMINATE, not positively healthy.

This PR changes health-state calculation only. Event/report policy is handled
separately.

Stack

Depends on #101. Review the final commit only:

6041e24 [dhcpmon]: Match DHCPv4 counters across health windows

Verification

Fresh exact-head Azure PR CI is pending. No local compilation is used.
The previously validated behavior avoided the pre-VLAN false positive,
detected a real post-VLAN outage, and handled counter clear during traffic.

Back port request

None. This targets master only.

Xichen96 added 3 commits July 26, 2026 16:39
Resolve nested Ethernet-to-PortChannel-to-VLAN membership and aggregate counters at each immediate parent without changing existing Dual-ToR attribution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Treat a self-mapped VLAN or PortChannel member as a root interface instead of creating a self aggregate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Copilot AI review requested due to automatic review settings July 26, 2026 07:02
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates sonic-dhcpmon’s DHCPv4 relay health calculation to track “unmatched” RX activity across monitoring windows (per message type) using absolute counter watermarks and a one-window TX-before-RX credit model, while also tightening counter-state synchronization so health sampling and DB writes operate on coherent snapshots.

Changes:

  • Add counter-state read/write locking primitives and use them to serialize health sampling/snapshot/DB-sync operations against packet callbacks.
  • Implement DHCPv4 per-message-type unmatched-window tracking (watermarks + adjacent-window TX credit) and update positive health evaluation to return INDETERMINATE for idle flows.
  • Write COUNTERS_DB from a stable in-memory snapshot (copy under lock, write outside lock).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/sock_mgr.h Introduces counter-state lock APIs and snapshot counter copy/update interfaces.
src/sock_mgr.cpp Implements counter-state locking and snapshot-based DB update plumbing.
src/packet_handler.cpp Binds packet callback work and wraps packet handling with counter-state read locking.
src/dhcp_mon.cpp Wraps health checks / cache sync / DB update paths with counter-state write locking and uses snapshot DB updates.
src/dhcp_device.h Adds APIs for DHCPv4 unmatched-window tracking and health-state reset.
src/dhcp_device.cpp Implements per-message-type unmatched-window tracking and updates positive health logic accordingly.

Comment thread src/dhcp_device.cpp
Comment thread src/sock_mgr.h Outdated
Treat configured context interfaces as roots and fail closed after the expected physical-to-PortChannel-to-VLAN depth, avoiding cyclic recursion in packet processing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Copilot AI review requested due to automatic review settings July 26, 2026 07:43
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/sock_mgr.h:142

  • sock_mgr_copy_cache_counters() reads all live per-socket counter maps and must be called under an exclusive counter-state lock to avoid races with packet callbacks incrementing counters. The header comment currently doesn’t document this requirement, which makes accidental unsafe use more likely.
/** Copy cache counters for all sockets */
socket_counters_t sock_mgr_copy_cache_counters();

Comment thread src/sock_mgr.cpp Outdated
Comment thread src/dhcp_mon.cpp Outdated
Copilot AI review requested due to automatic review settings July 26, 2026 07:51
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comment thread src/dhcp_device.cpp
Comment thread src/packet_handler.cpp Outdated
Copilot AI review requested due to automatic review settings July 26, 2026 08:05
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Comment thread src/dhcp_mon.cpp
Copilot AI review requested due to automatic review settings July 26, 2026 08:14
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

Xichen96 added 5 commits July 27, 2026 14:00
Keep hierarchy traversal in the device manager while placing the pure aggregate-name formatter with the existing aggregate utilities.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Document unmapped parent lookup and root aggregate fallback without changing behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Keep the established get_agg_counter_ifname utility name while changing only its hierarchy semantics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
@Xichen96
Xichen96 force-pushed the dev/xichenlin/track-unmatched-v4-windows branch from 0c0a31d to 7c63b1b Compare July 28, 2026 04:36
Copilot AI review requested due to automatic review settings July 28, 2026 04:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/dhcp_device.cpp:120

  • The new health algorithm reads sock_info.all_counters directly while packet-handler threads concurrently increment those counters. Since the counters are stored in std::unordered_map<uint8_t, uint64_t>, these read/write operations are not synchronized and constitute undefined behavior (data races and potentially torn 64-bit reads), which can lead to incorrect rx_delta/tx_delta calculations or crashes under load. Consider making counter values atomic or introducing a lock/snapshot mechanism so health evaluation reads a stable view of counters while packet threads update them.
    const sock_info_t &rx_sock_info = sock_mgr_get_sock_info(rx_sock);
    const counter_t &rx_counters = rx_sock_info.all_counters.at(ifname);
    const sock_info_t &tx_sock_info = sock_mgr_get_sock_info(tx_sock);
    const counter_t &tx_counters = tx_sock_info.all_counters.at(ifname);

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Xichen96 and others added 9 commits July 28, 2026 15:58
Document root-interface parent semantics and the private context traversal helper.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Populate one state per discovered VLAN or PortChannel check, derive aggregate ratios from topology, and format aggregate disparity errors with the interface and duration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Return indeterminate for same-message-type DHCPv6 relay health because client and relay message types differ across the relay boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Document that the disabled same-type check always returns indeterminate and takes no interface parameter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Require selected DHCPv6 client or nested-relay input to produce Relay-Forward output, and Relay-Reply input to produce a valid downstream reply type. Compare activity rather than magnitude so multiple configured servers remain valid.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Name the transformation subsets as monitored_v6 groups, restore explicit SARR terminology, and place the IPv4 and IPv6 positive-health functions together.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
State the exact DORA same-type relationship and DHCPv6 forward/reply transformation groups checked by the positive-health functions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
Track unmatched receive activity by message type so idle windows and adjacent transmit activity do not create false relay-loss state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 39f979be-d826-4d5c-949a-f20abb58bb83
Signed-off-by: Xichen96 <lukelin0907@gmail.com>
@Xichen96
Xichen96 force-pushed the dev/xichenlin/track-unmatched-v4-windows branch from 7c63b1b to 6041e24 Compare July 28, 2026 16:15
Copilot AI review requested due to automatic review settings July 28, 2026 16:15
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/dhcp_device.cpp:213

  • check_counter_not_transmitted() is now unused (only its definition remains). Leaving dead code here makes future refactors harder and can trigger -Wunused-function warnings under common build flags.

Remove the function if it’s no longer needed (or reintroduce a call site if it’s meant to back a health check).

static bool check_counter_not_transmitted(const std::string &ifname, int rx_sock, int tx_sock,
                                          const int *monitored_msgs, size_t monitored_msg_cnt)
{
    const sock_info_t &rx_sock_info = sock_mgr_get_sock_info(rx_sock);
    const counter_t &rx_counters = rx_sock_info.all_counters.at(ifname);
    const counter_t &rx_snapshot = rx_sock_info.all_counters_snapshot.at(ifname);
    const sock_info_t &tx_sock_info = sock_mgr_get_sock_info(tx_sock);
    const counter_t &tx_counters = tx_sock_info.all_counters.at(ifname);
    const counter_t &tx_snapshot = tx_sock_info.all_counters_snapshot.at(ifname);

    for (size_t i = 0; i < monitored_msg_cnt; i++) {
        int msg_type = monitored_msgs[i];
        if (rx_counters.at(msg_type) > rx_snapshot.at(msg_type) &&
            tx_counters.at(msg_type) <= tx_snapshot.at(msg_type)) {
            return true;
        }
    }
    return false;
}

@Xichen96

Copy link
Copy Markdown
Contributor Author

Closing because the persistent cross-window watermark model does not match the intended point-in-time health semantics. It will be replaced after the final relay transformation, downstream fan-out, and configured-server ratio rules.

@Xichen96 Xichen96 closed this Jul 28, 2026
@Xichen96
Xichen96 deleted the dev/xichenlin/track-unmatched-v4-windows branch July 28, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants