Skip to content

[dualtor]: Add IPv6 only active-active control plane design #2403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zjswhhh wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[dualtor]: Add IPv6 only active-active control plane design #2403

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d8965c5
[dualtor]: Add IPv6 only active-active control plane design
zjswhhh Jun 23, 2026
d0df1fa
minor
zjswhhh Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions doc/dualtor/active_active_ipv6_only_control_plane.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# IPv6-Only Active-Active Control Plane Support

## Revision

| Rev | Date | Author | Change Description |
| --- | --- | --- | --- |
| 0.1 | 2026-06-22 | Jing Zhang | Initial design for IPv6-only active-active DualToR control-plane support |

## Overview

This document describes the SONiC behavior required to support IPv6-only active-active DualToR clusters for control-plane paths that currently depend on IPv4 endpoints.

The change is intentionally conservative. Existing IPv4-only and dual-stack deployments must continue to use IPv4. IPv6 is selected only when the corresponding IPv4 field is missing.

This document does not redesign DualToR data-plane forwarding, neighbor handling, IPinIP routing, or the active-active state machine. Those behaviors should remain unchanged unless validation exposes a direct dependency on IPv4-only control endpoints.

## Background

Active-active DualToR deployments use linkmgrd to probe server or SoC reachability and use ycabled to send gRPC requests to the SoC or NIC side for forwarding-state control.

The `MUX_CABLE` schema already has both IPv4 and IPv6 endpoint fields:

```text
MUX_CABLE|<port>:
server_ipv4: <server IPv4 prefix>
server_ipv6: <server IPv6 prefix>
soc_ipv4: <SoC IPv4 prefix>
soc_ipv6: <SoC IPv6 prefix>
cable_type: active-active | active-standby
state: auto | manual | detach | active | standby
```

No new Config DB fields are required.

## Problem Statement

An IPv6-only active-active cluster may have `server_ipv6` and `soc_ipv6` for a mux cable, but no `server_ipv4` or `soc_ipv4`. Existing IPv4-oriented control paths can fail in this shape:

1. Config generation may skip a `MUX_CABLE` row when the server IPv4 loopback is missing, even if the server IPv6 loopback exists.
2. linkmgrd may fail to select an IPv6 endpoint for link probing when IPv4 is absent.
3. ycabled may fail to create a gRPC channel when `soc_ipv4` is absent, even if `soc_ipv6` exists.

As a result, an IPv6-only active-active cluster can miss required mux configuration, fail link probing, or fail SoC-side gRPC control.

## Goals

1. Support IPv6-only active-active DualToR control-plane endpoints.
2. Preserve IPv4 behavior for existing deployments.
3. Preserve IPv4 preference for dual-stack deployments.
4. Avoid Config DB schema changes.
5. Keep failures isolated to the affected port when an endpoint is missing or malformed.
6. Preserve existing active-active state-machine behavior.


## Component Requirements

| Component | Required behavior |
| --- | --- |
| Config generation | <ul><li>Generate `MUX_CABLE` entries when `server_ipv6` exists even if `server_ipv4` is absent.</li><li>Include `server_ipv6` and `soc_ipv6` fields when available.</li><li>Preserve IPv4 fields when they are present.</li></ul> |
| linkmgrd | <ul><li>Select IPv4 endpoints first when present.</li><li>Use `server_ipv6` and `soc_ipv6` when the matching IPv4 field is absent.</li><li>Use Loopback2 IPv6 as the link-prober source address when the selected probe endpoint is IPv6.</li><li>Use Loopback3 IPv6 as the GUID source component when the selected probe endpoint is IPv6.</li><li>Use ICMPv6 link probing for IPv6-selected endpoints.</li><li>Create IPv6 hardware ICMP echo session fields when hardware probing is enabled for IPv6-selected endpoints.</li></ul> |

@lolyu lolyu Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 These two loopback rules (Loopback2 IPv6 = prober source, Loopback3 IPv6 = GUID source) are correct — I verified them against the linkmgrd source and PR #344 — but the doc states them as bare facts. Two additions would help: (a) a brief why for these specific loopbacks so the convention isn't magic to a future reader/operator; and (b) the failure behavior when an IPv6-only deployment is missing Loopback2/Loopback3 IPv6 addresses. The Error Handling section covers missing endpoint fields but not missing source loopbacks, yet the Validation section (item 2) implies they're required config — so a "IPv6 selected but Loopback2/3 IPv6 source unavailable" row in Error Handling would close that gap.

| ycabled | <ul><li>Select `soc_ipv4` first when present.</li><li>Use `soc_ipv6` when `soc_ipv4` is absent.</li></ul> |

## Examples

IPv6-only active-active port:

```json
{
"MUX_CABLE": {
"Ethernet8": {
"state": "auto",
"cable_type": "active-active",
"server_ipv6": "fc02:1000::2/128",
"soc_ipv6": "fc02:1000::3/128"
}
}
}
```

Expected behavior:

1. Config generation emits the `MUX_CABLE` entry.
2. linkmgrd probes `fc02:1000::3` with ICMPv6.
3. ycabled connects gRPC to `[fc02:1000::3]:<port>`.

Dual-stack active-active port:

```json
{
"MUX_CABLE": {
"Ethernet8": {
"state": "auto",
"cable_type": "active-active",
"server_ipv4": "10.10.10.2/32",
"server_ipv6": "fc02:1000::2/128",
"soc_ipv4": "10.10.10.3/32",
"soc_ipv6": "fc02:1000::3/128"
}
}
}
```

Expected behavior:

1. linkmgrd continues to use `soc_ipv4` for probing.
2. ycabled continues to use `soc_ipv4` for gRPC.
3. IPv6 fields remain available but are not selected by default.

## Compatibility

| Config shape | linkmgrd selected endpoint | ycabled selected endpoint |

@lolyu lolyu Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The selection granularity should be stated explicitly here. The rule is "use IPv6 when the matching IPv4 field is absent" — but server_* and soc_* are independent, so this matrix only covers the soc_* axis. It doesn't show the server×soc cross-product: e.g. server_ipv4 present + soc_ipv4 absent (only soc_ipv6) would have linkmgrd probing over IPv4 while ycabled connects gRPC over IPv6 on the same port. Is that mixed-family case supported? Recommend either (a) adding a sentence that family is selected independently per endpoint (server vs soc) and the mixed case is valid, or (b) adding a constraint that a port's server and soc endpoints must use the same family. Right now it's an unspecified behavior a reader can't resolve from the doc.

| --- | --- | --- |
| `soc_ipv4` only | `soc_ipv4` | `soc_ipv4` |
| `soc_ipv4` and `soc_ipv6` | `soc_ipv4` | `soc_ipv4` |
| `soc_ipv6` only | `soc_ipv6` | `soc_ipv6` |
| Neither field present | unavailable | unavailable |

No migration is required for existing deployments.

## Error Handling

For each mux port:

1. If both IPv4 and IPv6 endpoint fields are missing, leave the endpoint unavailable for that port and log the condition.
2. If the selected endpoint is malformed, reject that endpoint for the affected port and log the condition.
3. If IPv4 is present but malformed, do NOT silently fall back to IPv6.

@lolyu lolyu Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 This rule — IPv4 present-but-malformed must NOT silently fall back to IPv6 — is the right call and well stated. It correctly distinguishes "IPv4 absent" (fall back to IPv6) from "IPv4 broken" (fail loudly, don't mask a misconfiguration), which is exactly the distinction that prevents a malformed-config split-brain from being papered over. Good that it's explicit.


## Validation

Validation should be covered through sonic-mgmt so the IPv6-only active-active topology is represented by the same testbed, simulator, and traffic-control framework used by existing DualToR testing.

Required sonic-mgmt adaptation:

1. Configuration examples and generated testbed inputs must support active-active mux ports that have `server_ipv6` and `soc_ipv6`, without `server_ipv4` or `soc_ipv4`.
2. Testbed and simulator configuration must carry Loopback2 IPv6 and Loopback3 IPv6 addresses so linkmgrd can use Loopback2 as the ICMPv6 probe source and Loopback3 as the GUID source component.
3. The ICMP responder must support ICMPv6 echo replies for link-prober validation while preserving the link-prober payload.
4. The NiC simulator and its control helpers must support IPv6 SoC endpoints. Simulator management paths that build HTTP or gRPC URLs must handle IPv6 addresses correctly, while preserving existing IPv4 behavior.
5. Existing IPv4-only and dual-stack sonic-mgmt coverage must remain in place.

Validation is complete when sonic-mgmt can load an IPv6-only active-active configuration, drive ICMPv6 link-prober health through the responder, exercise NiC simulator gRPC control over IPv6 endpoints, verify mux state transitions, and rerun dual-stack coverage to confirm IPv4 preference.