Skip to content

[GHA] Add workflow to assign per-company triage labels to PRs - #26772

Open
bhouse-nexthop wants to merge 3 commits into
sonic-net:masterfrom
bhouse-nexthop:triage-label-assignment
Open

[GHA] Add workflow to assign per-company triage labels to PRs#26772
bhouse-nexthop wants to merge 3 commits into
sonic-net:masterfrom
bhouse-nexthop:triage-label-assignment

Conversation

@bhouse-nexthop

@bhouse-nexthop bhouse-nexthop commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Description of PR

Summary:
Adds a GitHub Actions workflow that automatically labels each new pull request with two triage-<company> labels, to spread PR triage load evenly across Arista, Cisco, Microsoft and Nexthop.

Opened as a draft for evaluation — see the open questions at the bottom before this is considered for merge.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • New Test case
    • Skipped for non-supported platforms
  • Test case improvement

Back port request

  • 202311
  • 202405
  • 202411
  • 202505
  • 202511
  • 202512
  • 202605

Tracking issue/work item for backport/cherry-pick request: N/A
Failure type: N/A

Approach

What is the motivation for this PR?

PR triage currently has no systematic assignment. This distributes every incoming PR to two companies, so each PR has a clear set of owners for triage, and the load is spread evenly over time. A PR is never assigned to the company its author works for.

How did you do it?

Three files:

File Purpose
.github/workflows/assign_triage_labels.yml Workflow: pull_request_target (opened/reopened), weekly cron, and workflow_dispatch
.github/triage-assign/companies.yml Config: companies, org names, username suffixes, per-user overrides
.github/triage-assign/assign_triage_labels.py The assignment logic

No credentials required. Everything the workflow reads is public: the author map below, and the repo's own PRs via the default GITHUB_TOKEN. No repo or org secret needs to be provisioned before this can run.

Even distribution without tracking. The two companies are drawn from a random.Random seeded with sha256("<owner>/<repo>#<pr_number>"). This needs no persistent state, and is idempotent — re-running on the same PR always produces the same two companies, so the backstop scan can never reshuffle an existing assignment.

Author exclusion. The author's own company is removed from the draw. Affiliation is resolved in order:

  1. the overrides map in companies.yml, for people the author map has not caught up with, or whose username carries no company suffix
  2. sonic-tsc/sii_author_predict.csv — public, fetched anonymously over raw.githubusercontent.com
  3. a username suffix heuristic — -arista, -cisco/-csco, -microsoft/-msft, -nexthop

If the fetch fails, the workflow logs a warning and carries on with the suffix heuristic rather than erroring out.

Two properties of that CSV drive the parsing:

  • Others means "unknown", not "some company other than these four". 842 of its 1765 rows sit in that bucket, and 31 of those carry a company suffix (aajith-arista, bining-nexthop, balram-csco, tadams-msft, …). Treating a map hit as authoritative is correct for a real org — someone listed as Nvidia should not have a coincidental username ending exclude a company — but applying that to Others would silently drop all 31. Such values are listed under a new unknown_organizations key in companies.yml and filtered out at load, so the suffix heuristic still sees those authors.
  • Three authors appear twice with conflicting organizations, one under the placeholder value author_org_dup. The row with the highest Score wins.

Backstop. Besides firing on PR open, the workflow runs weekly and can be triggered manually to scan open PRs carrying no company label yet. This covers PRs opened while the workflow was failing or disabled, and allows labelling historic PRs. Manual runs take a dry_run input to preview assignments without applying them, and a scan_limit input to bound the scan.

How did you verify/test it?

  • Author resolution against the live CSV: 921 authors carry a known organization, of which 393 resolve to the four configured companies — Microsoft 137, Cisco 113, Arista 72, Nexthop 71. The other 528 (Nvidia, Broadcom, Google, Marvell, …) correctly exclude nothing. No CSV organization contradicts the suffix heuristic in any row, so the fallback never disagrees with the map. Spot-checked map hits, Others-plus-suffix fallbacks, per-user overrides, non-participating orgs, and unknown authors.
  • Distribution, over 40k synthetic PRs with all four companies eligible: 20000 / 20118 / 19907 / 19975 — each within 0.5% of an even share, and all six company pairs within ~2% of even. With a Cisco author excluded across 30k PRs, the remaining three split 20139 / 19923 / 19938.
  • Determinism: repeated draws for the same PR number are identical.
  • End-to-end, with a mocked GitHub client: both single-PR and scan modes, missing-label creation, skipping already-labelled PRs, and dry_run applying nothing.
  • flake8 --max-line-length=120 clean, matching this repo's pre-commit config.

Any platform specific information?

None — CI infrastructure only, no test or testbed code is touched.

Supported testbed topology if it's a new test case?

N/A

Documentation

No wiki update yet. Happy to add one describing what the triage-* labels mean and what the assigned companies are expected to do, if that is wanted.


Open questions / required before merge

  1. The triage-* labels do not exist in this repository yet. The workflow creates any that are missing on first run (colour c5def5, overridable per-company via label_color), which is why it requests issues: write.

  2. Suggested rollout: a manual run with dry_run: true first, to review assignments against real PR authors before anything is labelled.

  3. Company list is currently Arista, Cisco, Microsoft, Nexthop. Adding a company is a one-entry change to companies.yml.

  4. Which author map to use. sonic-tsc carries two. This uses sii_author_predict.csv; the older sii_author_map/author.csv is not suitable — its organization column is free-form GitHub profile text rather than curated data (cisco, Cisco Inc., cisco systems, MSFT, and one entry reading @Coinbase @Microsoft @Snapchat @linkedin @IBM … that substring matching would misattribute to Microsoft), 582 of its 1520 rows have a null organization, and its coverage of the four companies is far thinner:

    sii_author_predict.csv (used) sii_author_map/author.csv
    Arista 72 1
    Cisco 113 61
    Microsoft 137 129
    Nexthop 71 22

    An earlier revision of this PR read contributors.json from sonic-net/sonic-contributor-map instead. Its coverage is somewhat better still (84 / 140 / 153 / 72), but that repository is private: it needs a CONTRIBUTOR_MAP_TOKEN secret and sonic-net admin rights to provision one before the workflow can resolve anyone. The public CSV plus the suffix heuristic closes most of that gap at no operational cost. If the TSC would rather use the private map, switching back is a small change — happy to do it.

Adds a GitHub Actions workflow that labels each new pull request with two
"triage-<company>" labels, spreading triage load across Arista, Cisco,
Microsoft and Nexthop.

The two companies are drawn deterministically from a SHA-256 seed of
"<owner>/<repo>#<pr_number>", so no persistent state or tracking is needed
and re-running the workflow on the same PR always yields the same result.
Distribution is even: over 40k synthetic PRs each company lands within 0.5%
of an even share, and all six company pairs within ~2%.

The PR author's own company is excluded from the draw. Author affiliation is
resolved in order:

  1. an "overrides" map in companies.yml, for people the contributor map has
     not caught up with
  2. contributors.json from sonic-net/sonic-contributor-map
  3. a username suffix heuristic (e.g. "someone-arista" -> Arista)

Note that sonic-net/sonic-contributor-map is a private repository, so the
default GITHUB_TOKEN cannot read it. Reading it requires a
CONTRIBUTOR_MAP_TOKEN secret with read access to that repository. Without the
secret the workflow still runs, falling back to the suffix heuristic.

Besides running on pull_request_target, the workflow runs weekly and can be
triggered manually to scan open PRs that carry no company label yet. This
backstops PRs opened while the workflow was failing or disabled, and allows
labelling historic PRs. Manual runs accept a dry_run input to preview the
assignments without applying them.

The triage-* labels do not exist in the repository yet; the workflow creates
any that are missing on first run.

Signed-off-by: Brad House <bhouse@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.

Semgrep flagged three blocking findings on the new files:

- dynamic-urllib-use-detected: urllib supports 'file://' schemes, so a
  dynamic URL is treated as a file-read risk. Switch the contributor map
  fetch to requests, which is the remediation the rule itself suggests.
  requests is already an indirect dependency via PyGithub, but install it
  explicitly rather than rely on that.

- github-actions-mutable-action-tag (x2): pin actions/checkout and
  actions/setup-python to full commit SHAs, since tags can be silently
  repointed by the action owner. Other workflows in this repository still
  use mutable tags, but Semgrep only scans changed files, so new workflows
  are held to the stricter bar.

Verified the assignments are unchanged after the refactor, and that an
unset token, an HTTP 401 and an unreachable host all still fall back to the
username suffix heuristic without raising.

Signed-off-by: Brad House <bhouse@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.

Replace the private sonic-net/sonic-contributor-map contributors.json with
sii_author_predict.csv from sonic-net/sonic-tsc, which is public and needs no
credentials. This drops the CONTRIBUTOR_MAP_TOKEN secret, and with it the
dependency on sonic-net admin rights before the workflow can resolve authors.

The CSV files 842 of its 1765 rows under the organization "Others", which means
"unknown" rather than "a company other than these". Treating a map hit as
authoritative is right for Nvidia or Broadcom, but applying it to "Others"
would drop the 31 authors whose username carries a company suffix yet sit in
that bucket (aajith-arista, bining-nexthop, balram-csco, ...). Those
organizations are listed under the new unknown_organizations key in
companies.yml and filtered out at load, so the suffix heuristic still sees
them.

Three authors appear twice with conflicting organizations, one of them under
the placeholder value "author_org_dup"; the row with the highest Score wins.

Verified against the live CSV: 921 authors carry a known organization, of which
393 resolve to the four configured companies (Microsoft 137, Cisco 113, Arista
72, Nexthop 71), and no CSV organization contradicts the suffix heuristic.
Draw distribution and determinism are unchanged.

Signed-off-by: Brad House <bhouse@nexthop.ai>
@bhouse-nexthop
bhouse-nexthop marked this pull request as ready for review August 7, 2026 18:13
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants