Skip to content

[ci]: Complete sonic-swss CI-unification cutover - #4821

Draft
theasianpianist wants to merge 2 commits into
sonic-net:masterfrom
theasianpianist:ci-unification/phase2-swss-complete-cutover
Draft

[ci]: Complete sonic-swss CI-unification cutover#4821
theasianpianist wants to merge 2 commits into
sonic-net:masterfrom
theasianpianist:ci-unification/phase2-swss-complete-cutover

Conversation

@theasianpianist

Copy link
Copy Markdown
Contributor

Summary

Complete all migration changes owned by the sonic-swss repository in one PR,
following merged sonic-swss-common #1222 and sonic-sairedis #2016.

This PR moves SWSS Build and VS-test host setup to the shared
sonic-swss-common/ci/buildenv_setup mechanism, makes sonic-swss the single
owner of the reusable docker-sonic-vs build and VS-test templates, and cuts
SWSS's own BuildDocker/Test stages over to those templates.

azure-pipelines-dash.yml is deprecated and intentionally unchanged.
Cross-repo references from sonic-swss-common/sonic-sairedis and the final
required shared-infra gate remain separate follow-ups because they modify other
repositories/policy.

Build and local development

  • Add declarative build-env/packages/{base,tooling,test}.yaml and
    build-env/upstream-artifacts.yaml.
  • Declare only immediate Build artifacts (sonic-sairedis + dash-api); the
    sairedis artifact recursively supplies swss-common/common-libs/VPP and shared
    package/post-install configuration.
  • Replace inline apt/artifact/Redis/VPP setup in the complete Build matrix with
    shared buildenv_setup.
  • Add canonical build-env/build.sh for package build + Rust tests.
  • Delete the hardcoded local build-env/container-setup.py artifact installer.
  • Preserve the existing non-root/home-mounted/custom-setup local Docker workflow,
    but drive its canonical dependencies through buildenv_setup.
  • Publish build-env/ additively beside the existing top-level DEBs and coverage
    files for future downstream cascading.
  • Expose repo + staged_upstreams on the Build template for same-run reuse by
    upstream pipelines.

Shared docker-sonic-vs build

  • Replace build-docker-sonic-vs-template.yml with reusable
    build-docker-template.yml.
  • Parameterize consumer differences: debug package, VPP, gcov source handling,
    nexthopgroup, and current-run swss-common/sairedis artifacts.
  • Keep SWSS's current behavior as the defaults, including ASAN image tagging,
    VPP sysctl handling, coverage-source restoration, branch/result filters, and
    package set.
  • Parameterize the swss-owned Dockerfile so sairedis and swss-common can reference
    the same implementation in later PRs.

Shared VS-test execution

  • Replace test-docker-sonic-vs-template.yml with reusable
    run-vs-tests-template.yml.
  • Use buildenv_setup --scope test plus SWSS-owned packages/test.yaml and
    test-scoped artifact declarations for the Ubuntu 22.04 host.
  • Preserve kernel-module setup, Redis configuration, Docker image loading,
    retries/parallelism, JUnit, logs, coverage, ASAN reporting, VRF smoke check,
    and SWSS's single_asic_voq_fs test pass.
  • Provide neutral defaults, repo, staged_upstreams, extra_modes, and
    explicit failure-policy parameters for later sairedis/swss-common adoption.

Validation completed before PR

  • YAML parsing, shell syntax, Docker Compose config, git diff --check.
  • Dockerfile docker build --check.
  • buildenv_setup --dry-run for bookworm/trixie across amd64/armhf/arm64 and
    Ubuntu 22.04 Test scope.
  • Matrix assertions for conditional package/artifact selection.
  • Synthetic end-to-end Build cascade:
    swss -> sairedis -> common-libs/swss-common/sonic-yang/VPP + dash-api,
    including required-staged enforcement, VPP's isolated install environment,
    and shared Redis source resolution.
  • Synthetic Test cascade for common-libs, swss-common Ubuntu test-host and
    sonic-buildimage Ubuntu artifacts.
  • Real Build setup in sonic-slave-bookworm:master-amd64, using sairedis PR
    artifact 1183201 as a required staged bundle; all nested artifacts and the
    shared Redis hook installed successfully.
  • Real Test setup in Ubuntu 22.04; libyang/common-libs, swss-common host DEBs,
    protobuf/dash DEBs, Python libyang binding and Redis setup completed. This run
    caught and fixed the YAML quoting of --force-confask,confnew (d66a8129).
  • Independent high-confidence review reported no remaining significant issues.

Temporary CI note

Post-merge sairedis master run 1186981 (source 24673778) is still in
progress. buildenv_setup intentionally resolves only completed upstream runs,
so the first Azure run may select a pre-migration sairedis master artifact that
does not publish build-env/. Once 1186981 completes, Azure CI will be
retriggered and must use the cascade-ready master artifact across bookworm,
arm, and trixie legs.

Design context: sonic-net/SONiC#2419.

theasianpianist and others added 2 commits August 7, 2026 19:02
Land all sonic-swss-owned migration surfaces in one change.

Build and local development:
- replace the local container-setup.py artifact installer and duplicated
  Build-template setup with sonic-swss-common's shared buildenv_setup
- add declarative Build/Test packages and immediate upstream artifacts;
  inherit sairedis -> swss-common -> common-libs/VPP transitively
- add the canonical build.sh, preserve the existing non-root/custom local
  development workflow, and publish build-env/ additively with the existing
  top-level DEBs/coverage artifacts
- expose repo + staged_upstreams so upstream pipelines can reuse SWSS's
  canonical Build template with required same-run artifacts

Shared docker-sonic-vs stack:
- replace the repo-specific BuildDocker template with a swss-owned reusable
  build-docker-template
- parameterize debug package, VPP, gcov, nexthopgroup, and current-run
  swss-common/sairedis artifact sources while keeping SWSS behavior default
- parameterize the Dockerfile without changing its current package set,
  coverage source handling, or ASAN image behavior

Shared VS test stack:
- replace the imperative local test template with run-vs-tests-template
- use buildenv_setup --scope test and packages/test.yaml for the shared host
  environment, while retaining module setup, coverage, ASAN, logging, JUnit,
  retry/parallel execution, and the SWSS single_asic_voq test pass
- provide neutral defaults plus staged_upstreams/repo parameters for later
  sonic-sairedis and sonic-swss-common adoption

The deprecated azure-pipelines-dash.yml is intentionally unchanged. Consumer
repo references and the final required shared-infra gate remain follow-up PRs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19e0b615-271d-48f0-98cc-7ad6905c7f9e
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Quote --force-confask,confnew in YAML so it is passed to dpkg as a single
option rather than being split into an option plus a nonexistent `confnew`
archive path.

Validated the corrected test scope end-to-end in Ubuntu 22.04: common-libs,
swss-common test-host and sonic-buildimage artifacts install successfully,
libyang Python bindings build, and the shared Redis hook completes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19e0b615-271d-48f0-98cc-7ad6905c7f9e
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@theasianpianist

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

@theasianpianist

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants