[vpp] SAIVPP unit-test harness (docker-sai-test-vpp) + devdocs by nicholasching · Pull Request #1950 · sonic-net/sonic-sairedis

nicholasching · 2026-06-23T16:04:13Z

Description of PR

Summary:
Adds the self-contained Docker harness (docker-sai-test-vpp) that runs the upstream OCP sai_test PTF suite against the real VPP SAI backend (libsaivs) in one container — VPP + saiserver + PTF + veth/AF_PACKET topology — and produces a per-test compatibility matrix. This is test scaffolding only (everything under .azure-pipelines/docker-sai-test-vpp/); it does not touch any production/runtime code, so it cannot affect existing sonic-sairedis functionality. Reviewers can start at .azure-pipelines/docker-sai-test-vpp/README.md and run_test.sh.

Fixes # (N/A — no upstream issue)

Type of change

Approach

What is the motivation for this PR?

There was no in-tree way to run the OCP sai_test suite against the VPP SAI backend and track per-test coverage. This harness provides that, plus the development docs and compatibility matrices used to drive the backend fixes.

How did you do it?

Added under .azure-pipelines/docker-sai-test-vpp/:

Dockerfile and run_test.sh (orchestrates Redis → veth/PortChannel topology → VPP → saiserver → PTF), plus T0 config (sai.profile, lanemap.ini, port-map.ini, ptf-port-map.ini) and gen_compatibility_matrix.py.
Config-signature grouping: because the VPP SAI backend builds the switch once per saiserver process, run_test.sh groups tests by their setUp config signature and restarts the backend per group, so one container can run any mix of tests.
ISOLATE_EACH_TEST (default on): run each test in its own group with a fresh backend + own config to eliminate cross-test config-reuse contamination; set 0 for faster grouped runs.
L3-over-LAG forwarding (harness portion): run_test.sh assigns the DUT-side connected IPs to each LAG (be<N>) and SVI (bvi<vlan>) so routed-to-LAG/SVI traffic forwards in the standalone PTF environment (the backend portion is in the backend-fixes PR).
Debug hold to keep VPP/saiserver/veths alive after a --debug run for vppctl inspection.
devdocs/, demodocs/, and results/ compatibility matrices; .gitignore for generated artifacts.

How did you verify/test it?

Running the harness against sai_route_test/sai_rif_test/sai_neighbor_test/sai_ecmp_test produces a compatibility matrix (current state: 34 PASS, captured in devdocs//results/). The harness is self-contained and does not build or run any production code.

Any platform specific information?

VPP platform test infrastructure only; isolated under .azure-pipelines/docker-sai-test-vpp/. No impact on the sonic-sairedis build or runtime behavior.

Documentation

This PR adds the harness README.md plus rolling/dated progress notes and deep-dive debug logs under .azure-pipelines/docker-sai-test-vpp/devdocs/ and demo docs under demodocs/.

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

…test Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

…dog, --relax Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

…ature and restarting backend per group Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

…ugging Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

UT-harness portion of the L3-over-LAG forwarding fix: run_test.sh assigns the DUT-side connected IPs to each LAG (be<N>) and SVI (bvi<vlan>) so routed-to-LAG /SVI traffic forwards in the standalone PTF environment. The backend portion (SwitchVppFdb.cpp) is in the VPP SAI backend PR. Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

linux-foundation-easycla · 2026-06-23T16:04:20Z

The committers listed above are authorized under a signed CLA.

✅ login: nicholasching / name: Nicholas Ching (01a0762, 02b11fa, 120e57f, 145aec2, 15776bb, 2661943, 4cd92d8, 4e1091b, 5780b35, 5cda18e, 77a77e9, 82ca412, b2ee2a3, b34a713, b882a30, c36a5f1, c78809e, dcc9a8f)

mssonicbld · 2026-06-23T16:04:21Z

/azp run

azure-pipelines · 2026-06-23T16:04:35Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-06-23T17:59:21Z

/azp run

azure-pipelines · 2026-06-23T17:59:33Z

Azure Pipelines successfully started running 1 pipeline(s).

…SER) - gen_compatibility_matrix.py: prefer defusedxml for parsing PTF JUnit XML (XXE-hardened), with a stdlib fallback; input is our own local test output. - Dockerfile: annotate the root entrypoint with a nosemgrep justification — the --privileged harness must run as root to start VPP/Redis/saiserver and manage veth/AF_PACKET; it is a disposable test container, never a deployed service. Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-06-23T18:04:03Z

/azp run

azure-pipelines · 2026-06-23T18:04:16Z

Azure Pipelines successfully started running 1 pipeline(s).

Satisfy tests/checkwhitespace.sh (Azure Build amd64 gate). Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-05T17:59:20Z

/azp run

azure-pipelines · 2026-08-05T17:59:33Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated no new comments.

Suppressed comments (4)

vslib/vpp/SwitchVppRif.cpp:359

getPortHwifNameFromLane uses a fixed 8-entry buffer for SAI_PORT_ATTR_HW_LANE_LIST and treats any non-success as “lane list unavailable”. If the port ever reports more lanes than fit, the call will typically fail with SAI_STATUS_BUFFER_OVERFLOW, and this path will silently fall back to TAP mapping (or fail later) without a clear error. Handle BUFFER_OVERFLOW explicitly and include the status code in the debug log so failures are diagnosable.

    uint32_t lanes[8] = {};
    sai_attribute_t attr = {};
    attr.id = SAI_PORT_ATTR_HW_LANE_LIST;
    attr.value.u32list.count = sizeof(lanes) / sizeof(lanes[0]);
    attr.value.u32list.list = lanes;

.azure-pipelines/docker-sai-test-vpp/run_test.sh:1012

run_one_ptf hard-codes the "Turn up ports..." marker string instead of using the LINK_UP_TRIGGER variable defined near the top of the script. This makes LINK_UP_TRIGGER effectively dead configuration and can confuse readers expecting it to work.

        case "$ptf_line" in
            *"Turn up ports..."*)
                bring_up_veths
                ;;
        esac

.azure-pipelines/docker-sai-test-vpp/run_test.sh:566

enumerate_test_classes() is currently defined but never called (planning uses plan_test_groups() instead). Keeping an unused discovery implementation in a large script makes future maintenance harder (two code paths to keep consistent).

# Enumerate every test class under $SAI_TEST_DIR as "module.Class" selectors so
# each can run as its own config-reuse ptf invocation. Best-effort: on any
# failure prints nothing and the caller falls back to a single invocation.
enumerate_test_classes()
{

vslib/vpp/SwitchVppRif.cpp:346

PR description states the change is “test scaffolding only (everything under .azure-pipelines/docker-sai-test-vpp/)” and “does not touch any production/runtime code”, but this PR also modifies vslib/vpp (e.g., adds lane-based interface resolution in SwitchVppRif.cpp / SwitchVpp.h). Please update the description to reflect the vslib runtime change, or split runtime changes into a separate PR if the intent is to keep this PR harness-only.

bool SwitchVpp::getPortHwifNameFromLane(
      _In_ sai_object_id_t port_id,
      _Out_ std::string& if_name)
{
    SWSS_LOG_ENTER();

mssonicbld · 2026-08-05T18:31:08Z

/azp run

azure-pipelines · 2026-08-05T18:31:22Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T14:00:59Z

/azp run

azure-pipelines · 2026-08-07T14:01:12Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T14:34:20Z

/azp run

azure-pipelines · 2026-08-07T14:34:32Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T15:50:31Z

/azp run

azure-pipelines · 2026-08-07T15:50:43Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T18:46:38Z

/azp run

azure-pipelines · 2026-08-07T18:46:50Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T20:14:47Z

/azp run

azure-pipelines · 2026-08-07T20:14:58Z

Azure Pipelines: Successfully started running 1 pipeline(s).

yue-fred-gao · 2026-08-07T21:20:32Z

+
+        if (get(SAI_OBJECT_TYPE_PORT, port_id, 1, &attr) != SAI_STATUS_SUCCESS)
+        {
+            SWSS_LOG_DEBUG("lane list unavailable for port %s",


LANE_LIST is mandatory in PORT. This should be an error message. Same for all the DEBUG message in the returning false case in this function.

yue-fred-gao · 2026-08-07T21:27:18Z

+    }
+
+    if_name = mapped_hwifname;
+    SWSS_LOG_DEBUG("resolved port %s lane set to %s/%s",


use INFO. DEBUG level is overwhelmed in syncd due to SWSS_LOG_ENTER in every function.

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

mssonicbld · 2026-08-07T22:04:56Z

/azp run

azure-pipelines · 2026-08-07T22:05:08Z

Azure Pipelines: Successfully started running 1 pipeline(s).

nicholasching · 2026-08-08T02:16:23Z

/azpw run

mssonicbld · 2026-08-08T02:16:26Z

⚠️ Notice: /azpw run only runs failed jobs now. If you want to trigger a whole pipline run, please rebase your branch or close and reopen the PR.
💡 Tip: You can also use /azpw retry to retry failed jobs directly.

Retrying failed(or canceled) jobs...

mssonicbld · 2026-08-08T02:16:28Z

Retrying failed(or canceled) stages in build 1187084:

✅Stage BuildAsan:

Job amd64: retried.

nicholasching added 17 commits June 23, 2026 07:49

feat: configuration for 32 ports; phase 1, task 2

77a77e9

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

feat: configuration and run script; phase 1, task 3-4; fails to pass …

b2ee2a3

…test Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

fix: ifmap prefix, IPv6 disable, PortChannel emulation, veth-up watch…

145aec2

…dog, --relax Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

fix: run multiple tests sequentially by grouping tests by config sign…

4cd92d8

…ature and restarting backend per group Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

feat: script to generate compatabilty matrix from test results

2661943

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: add feature development progress context

82ca412

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: add build and usage instructions for UT harness

15776bb

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

chore: add hold when debug enabled to leave container running for deb…

dcc9a8f

…ugging Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: add demo documentation

c36a5f1

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: add coverage by functionality

b882a30

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: minor updates to demo doc

01a0762

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: current dev state, compatibility matrix

c78809e

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: LAG-RIF neighbor/route + egress-disable progress (6-18)

4e1091b

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

feat: harness per-test isolation mode, restart backend between tests

5780b35

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: update to span full width

b34a713

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

docs: neighbor host-route local0 fix progress (6-19)

5cda18e

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

This was referenced Jun 23, 2026

[vpp] Dedicated binary-API socket and thread for asynchronous VPP events #1951

Merged

[vpp] VPP SAI backend fixes: CPU queue, host-interface admin-up, L3-over-LAG/host-route forwarding, LAG egress-disable #1952

Merged

nicholasching force-pushed the vpp_ut_harness branch from 73630af to 02b11fa Compare June 23, 2026 17:59

nicholasching force-pushed the vpp_ut_harness branch from 02b11fa to 4618cda Compare June 23, 2026 18:03

chore: strip trailing whitespace in progress-6-10 devdoc

7805b63

Satisfy tests/checkwhitespace.sh (Azure Build amd64 gate). Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

Copilot AI review requested due to automatic review settings August 5, 2026 17:59

Copilot started reviewing on behalf of nicholasching August 5, 2026 17:59 View session

Copilot AI reviewed Aug 5, 2026

View reviewed changes

nicholasching force-pushed the vpp_ut_harness branch from 739e5de to be770b3 Compare August 5, 2026 18:31

fix: resolve VPP interfaces through SONiC port config

018e8b2

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

fix: make port config comment spell-check clean

3651e36

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

fix(vpp): own port dependency initialization

d3f1bd9

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

refactor(vpp): own port config parsing

c9956c8

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

revert(vslib): restore upstream default port creation order

07e374f

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

yue-fred-gao reviewed Aug 7, 2026

View reviewed changes

fix: log lane resolution failures at error and success at info level

21a65e5

Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>

Uh oh!

Conversation

nicholasching commented Jun 23, 2026

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Documentation

Uh oh!

linux-foundation-easycla Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssonicbld commented Jun 23, 2026

Uh oh!

azure-pipelines Bot commented Jun 23, 2026

Uh oh!

mssonicbld commented Jun 23, 2026

Uh oh!

azure-pipelines Bot commented Jun 23, 2026

Uh oh!

mssonicbld commented Jun 23, 2026

Uh oh!

azure-pipelines Bot commented Jun 23, 2026

Uh oh!

mssonicbld commented Aug 5, 2026

Uh oh!

azure-pipelines Bot commented Aug 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

mssonicbld commented Aug 5, 2026

Uh oh!

azure-pipelines Bot commented Aug 5, 2026

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

yue-fred-gao Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

yue-fred-gao Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Aug 7, 2026

Uh oh!

azure-pipelines Bot commented Aug 7, 2026

Uh oh!

nicholasching commented Aug 8, 2026

Uh oh!

mssonicbld commented Aug 8, 2026

Uh oh!

mssonicbld commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

linux-foundation-easycla Bot commented Jun 23, 2026 •

edited

Loading