Add pr vs triage agent to analyze pr vstest failures

[lolyu]

Description of PR

Summary:
Fixes # (issue)

Type of change

• Bug fix
• New feature
• Refactor / cleanup
• Documentation update
• Test improvement

Approach

What is the motivation for this PR?

This is inspired from fixing recent PR vstest failures.

VS-test (Test / TestAsan) failures on sonic-sairedis PRs are almost always
recurring sonic-swss test flakes, but every reviewer re-derives that
conclusion by hand: fetch ADO logs, download the log@1 artifact, read
syslog / swss.rec / sairedis.rec, check whether the PR diff can even
reach the failing code path, and correlate against recent unrelated PRs.
This is repetitive, error-prone (PRs have been blamed for failures their
diff cannot have caused, e.g. VPP-only diffs vs vs-backend tests), and
the institutional knowledge (Known Flake Catalog, log layout, retry
detection, ADO REST endpoints) only lives in chat history.

This PR commits that knowledge as an agent spec so the triage is
reproducible and the lessons learned are version-controlled.

Work item tracking

• Microsoft ADO (number only): 38615568

How did you do it?

Added a single new file:

• .github/agents/pr-vs-triage.agent.md (+537 / -0)

The document is organized as:

• Scope — what the agent triages (Test / TestAsan only) and what it
  explicitly refuses (Build*, BuildSwss, BuildDocker*, GH Actions,
  infra-only failures).
• Pipeline background — stage list and which ones map to which check-run
  names on GitHub.
• Triage workflow — step-by-step recipe:
  1. Identify the PR.
  2. List checks; filter to in-scope failures.
  3. Pull ADO logs anonymously (mssonic/build project, public-read),
     including the per-test log@1 artifact with syslog, swss.rec,
     sairedis.rec, and sai_failure_dump/.
  4. PR-diff scoping (table mapping modified paths to whether they reach
     the vs backend) — frees the reviewer from blaming structurally
     unreachable failures.
  5. Cross-build correlation against the last 4–5 builds.
  6. Classification (swss-vs-test-fail, swss-vs-flake, asan-leak|uaf,
     unknown).
  7. Report block + verdict + optional unified-diff fix proposal targeting
     sonic-swss/tests/.
• Local repro recipes (docker-sonic-vs.gz, team kernel module build,
  draft sairedis PR pinning a swss fork branch).
• Lessons learned — running list of gotchas (retry directories clobbered,
  exit code 123 semantics, anonymous fallback endpoints, etc.).
• Hard rules — read-only; no rerun / merge / comment; quote logs verbatim;
  stop on auth-gated or out-of-scope failures.

No source code, build, packaging, or CI files are touched. There are no
runtime dependencies.

How did you verify/test it?

N/A

Any platform specific information?

N/A

Documentation

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[yxieca]

Summary:

• Adds a documentation file (.github/agents/pr-vs-triage.agent.md) defining an AI agent spec for triaging VS test failures on sonic-sairedis PRs. No code/CI behavior changes; purely docs.

Reviewed by: AI agent on behalf of Ying.

Non-blocking nits:

• Consider a brief validation note (e.g., tested against a recent failed build) since the doc includes specific ADO commands/URLs.
• Some hard-coded ADO GUIDs/URLs may be brittle; add guidance to re-derive or mention fallbacks (e.g., artifact zip).
• Minor inconsistencies/redundancy: repeated read-only constraints, placement of “download artifact” step, inconsistent “vstest/VS test” terminology, minor grammar.
• The referenced “Known Flake Catalog” isn’t included; add a starter catalog or remove the reference.
• Optional: add a lightweight version/date header or clarify .github/agents/.agent.md consumption conventions.

[Gfrom2016]

Good spec overall — the PR-diff scoping table (§4a) and cross-build correlation workflow are the strongest parts and directly solve the "innocent PRs getting blamed" problem.

A few things worth addressing:

Substantive:

1. "Known Flake Catalog" is referenced but never defined. Section 5 uses it as a classification signal but there’s no catalog anywhere in the file or repo. Either add a starter list inline (even just a handful of well-known recurring flaky tests like PortChannel LACP, p4rt redis-socket), or remove the reference and rely solely on the cross-build correlation in §4b which is already well-defined.

2. Step 4 (download docker-sonic-vs artifact) is misplaced. Downloading the full docker image belongs in the "Local repro" section, not in the log-fetching triage workflow — it could cause an agent to pull ~1 GB unnecessarily when it only needs task logs.

3. No validation note. "How did you verify/test it?" says N/A. A quick note like "validated against build #XXXXXX for PR #YYYY" would confirm the curl/timeline/log@1 paths actually work as described.

Minor:

• Terminology: "vstest", "VS test", "VS-test" used interchangeably — pick one
• Read-only rule is repeated 4 times; once in Hard Rules is enough
• ADO project GUID is hardcoded in Lesson 4 — worth adding a note that it can be derived directly from the check link URL
• log@1 size warning (Lesson 10) should also appear inline at the §3e download step since an agent might happily pull ~1 GB mid-triage

Also worth triggering a /azp run — the pipeline was cancelled and hasn’t had a fresh run since May 21.

[lolyu]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[lolyu]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[lolyu]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[lolyu]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).