Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .github/workflows/full-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ jobs:
RUSTDOCFLAGS: -D warnings

msrv:
needs: lint
timeout-minutes: 25
name: MSRV (1.89)
runs-on: ubuntu-latest
Expand All @@ -102,6 +103,7 @@ jobs:
- run: cargo check --workspace --all-targets

docs-and-demo:
needs: shells
timeout-minutes: 25
name: generated docs + demo
runs-on: ubuntu-latest
Expand Down Expand Up @@ -205,6 +207,7 @@ jobs:
run: python3 scripts/check_skips.py --job ci/backends-ha --report "$RUNNER_TEMP/skips-ha.jsonl"

deploy:
needs: msrv
timeout-minutes: 20
name: deployment files
runs-on: ubuntu-latest
Expand Down Expand Up @@ -317,7 +320,7 @@ jobs:
# `sdk` job above runs the full checks on one interpreter; here only the test suites run on the others.
sdk-runtimes:
timeout-minutes: 20
needs: tools
needs: [tools, sdk]
name: client SDK tests (${{ matrix.runtime }})
runs-on: ubuntu-latest
env:
Expand Down Expand Up @@ -368,7 +371,7 @@ jobs:

orchestrator:
timeout-minutes: 30
needs: tools
needs: [tools, sdk-runtimes]
name: agent orchestrator (Go)
runs-on: ubuntu-latest
env:
Expand Down Expand Up @@ -434,6 +437,7 @@ jobs:
# The completion scripts must parse in the shells they are for. A missing shell fails the run (AG_REQUIRE_COMPONENTS=shell)
# instead of being skipped; PowerShell is on the runner image, zsh and fish are installed here.
shells:
needs: deploy
timeout-minutes: 20
name: shell completions (bash, zsh, fish, PowerShell)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -464,7 +468,7 @@ jobs:
# and the AG_LLM_MODEL variable; forks and pull requests from forks have neither, so the step is skipped).
mcp:
timeout-minutes: 20
needs: tools
needs: [tools, orchestrator]
name: MCP server with an agent (scripted; live model opt-in)
runs-on: ubuntu-latest
env:
Expand Down Expand Up @@ -495,6 +499,7 @@ jobs:
# `str | None` annotations broke scripts/e2e_smoke.py there). The interpreter comes from uv (already pinned
# above); the scripts that read TOML (tomllib, 3.11+) are tested by the jobs that need them.
scripts-py39:
needs: lint
timeout-minutes: 10
name: lightweight scripts on Python 3.9
runs-on: ubuntu-latest
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/qualification.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@ jobs:
secrets: inherit

web:
needs: codeql
uses: ./.github/workflows/web.yml
with:
full: true

security:
needs: codeql
uses: ./.github/workflows/security.yml
with:
full: true
Expand All @@ -38,9 +40,11 @@ jobs:
security-events: write

containers:
needs: [core, web, security]
uses: ./.github/workflows/containers.yml

docs:
needs: [core, web, security]
uses: ./.github/workflows/docs.yml
with:
full: true
Expand All @@ -50,21 +54,27 @@ jobs:
id-token: write

packaging:
needs: [core, web, security]
uses: ./.github/workflows/packaging.yml

coverage:
needs: [core, web, security]
uses: ./.github/workflows/quality.yml

windows:
needs: [core, web, security]
uses: ./.github/workflows/windows.yml

fuzz:
needs: [core, web, security]
uses: ./.github/workflows/fuzz.yml

performance:
needs: [core, web, security]
uses: ./.github/workflows/perf.yml

oci:
needs: [core, web, security]
uses: ./.github/workflows/oci-runtimes.yml

complete:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ jobs:
comment-summary-in-pr: never

sbom:
needs: [cargo-deny, npm-audit, pip-audit, govulncheck, hygiene]
if: inputs.full
name: SBOMs build (scripts/sbom.sh)
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Normal PRs and main commits use path-aware fast CI: Linux Rust unit/core-storage contracts, SDK checks against one shared server build when API/SDK files change, Go contracts when the executor changes, and offline workflow checks when tooling changes. Web changes run lint, types, unit tests and a production build; documentation changes build the site. Unrelated changes do not start Rust or cross-platform builds. On main, selection covers all changes since the last successful run, so cancelling an earlier run cannot drop its pending checks. Unknown paths or unavailable Git/API history conservatively select all fast components.

The full matrix runs through `qualification.yml` on demand, nightly, and during release preparation: macOS, Windows, Linux coverage (which also executes the full Linux suite), PostgreSQL/S3/HA, all SDK runtimes, CLI examples, all browser engines/viewports, CodeQL, fuzzing, OCI runtimes, packaging and performance checks. SDK/runtime jobs share one verified server/CLI/MCP build. Coverage supplies Linux full-suite correctness, so it is not also run as a separate identical Linux test job.
The full matrix runs through `qualification.yml` on demand, nightly, and during release preparation: macOS, Windows, Linux coverage (which also executes the full Linux suite), PostgreSQL/S3/HA, all SDK runtimes, CLI examples, all browser engines/viewports, CodeQL, fuzzing, OCI runtimes, packaging and performance checks. SDK/runtime jobs share one verified server/CLI/MCP build. Background qualification is staged and matrices are bounded to leave runner capacity for ordinary PRs. Coverage supplies Linux full-suite correctness, so it is not also run as a separate identical Linux test job.

Stale validation runs are cancelled. Explicitly pinned release candidates use separate refs and continue while main advances. Release publication promotes prepared, attested bytes and does not add a third full Rust run. See [Releasing](../operations/releasing.md) for the ready-candidate contract and publication time budget.

Expand Down
13 changes: 12 additions & 1 deletion scripts/test_ci_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,22 @@ def test_linux_full_tests_are_already_the_coverage_run(self):
def test_shared_tools_are_built_once_for_sdk_runtime_matrix(self):
jobs = workflow('full-ci')['jobs']
for name in ('sdk', 'sdk-runtimes', 'orchestrator', 'mcp'):
self.assertEqual(jobs[name]['needs'], 'tools')
self.assertIn('tools', [jobs[name]['needs']] if isinstance(jobs[name]['needs'], str) else jobs[name]['needs'])
commands = '\n'.join(s.get('run', '') for s in jobs[name]['steps'])
self.assertNotIn('cargo build', commands)
self.assertIn('sha256sum --check', commands)

def test_background_qualification_leaves_runner_headroom(self):
jobs = workflow('qualification')['jobs']
for name in ('web', 'security'):
self.assertEqual(jobs[name]['needs'], 'codeql')
for name in ('containers', 'docs', 'packaging', 'coverage', 'windows', 'fuzz', 'performance', 'oci'):
self.assertEqual(jobs[name]['needs'], ['core', 'web', 'security'])
core = workflow('full-ci')['jobs']
self.assertEqual(core['sdk-runtimes']['needs'], ['tools', 'sdk'])
self.assertEqual(core['orchestrator']['needs'], ['tools', 'sdk-runtimes'])
self.assertEqual(core['sdk-runtimes']['strategy']['max-parallel'], 2)

def test_browser_matrix_is_preserved_but_explicit(self):
jobs = workflow('web')['jobs']
for job in ('server-build', 'e2e', 'identity-provider'):
Expand Down
Loading