Skip to content

Commit c8382cb

Browse files
ClaudiaFangclaude
andcommitted
feat(e2e): multi-run isolation for real-provider E2E (Phase 2)
Centralize branch-namespace identity in scripts/e2e-namespace.sh (e2e/pr/<n>/<provider>/run-<id>-<attempt> and e2e/branch/<sanitized-id>/<provider>/run-<id>-<attempt>, with a content-hash suffix so differently-slashed branch names can't collide), sourced by e2e-harness.sh, the new e2e-namespace-cleanup.sh, and e2e-janitor.sh -- one canonical implementation, not several drifting apart. Implement the three-layer cleanup hierarchy the isolation model needs: - Layer 1 (scripts/e2e-harness.sh cleanup): unchanged in spirit, now deletes only this run's own uniquely-named branch. - Layer 2 (scripts/e2e-namespace-cleanup.sh + new .github/workflows/e2e-pr-cleanup.yml / e2e-branch-cleanup.yml): authoritative on PR close or source-branch delete, removes the whole e2e/pr/<n>/** or e2e/branch/<id>/** namespace. PR cleanup uses pull_request_target with no ref: override on checkout, so it only ever runs this repo's own trusted code/secrets, never the closing PR's branch. - Layer 3 (scripts/e2e-janitor.sh + new .github/workflows/e2e-janitor.yml, scheduled every 6h): TTL sweep (24h default) of any leftover e2e/** branch via generic git for-each-ref/push --delete, tolerant of already-deleted refs. Removes e2e-harness.sh's old ad hoc sweep subcommand/gfs-e2e-<provider>-* naming, superseded by the above. ci.yml's provider-e2e job: E2E_WORKDIR now pinned per run-id/run-attempt/provider under (was a shared e2e-<provider> dir), E2E_PR_NUMBER/E2E_SOURCE_BRANCH passed through for provision, and a per-source/provider concurrency group (cancel-in-progress: true) so a repeated push/rerun cancels its own predecessor -- cancellation is not a cleanup mechanism, so this is only possible because every run still gets its own unique branch regardless. Rewrites docs/testing/real-provider-e2e.md's isolation model with a Mermaid diagram of the cleanup hierarchy. Verification: npx eslint . -- 0 errors; npm run build -- clean; npx vitest run -- 527 passed; yaml.safe_load on all touched/new workflow files; bash -n on all new/changed scripts; functional dry-runs of the namespace/janitor/cleanup logic against throwaway local git repos; real end-to-end run against a live local Gitea sandbox (14/14 E2E tests passed) with the new harness code. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 4c4b04f commit c8382cb

10 files changed

Lines changed: 747 additions & 47 deletions

File tree

.github/workflows/ci.yml

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ jobs:
5050
- 'src/utils/symlink.ts'
5151
- 'e2e/**'
5252
- 'scripts/e2e-harness.sh'
53+
- 'scripts/e2e-namespace.sh'
54+
- 'scripts/e2e-namespace-cleanup.sh'
5355
- 'scripts/run-e2e.sh'
5456
- 'package.json'
5557
- 'package-lock.json'
@@ -80,6 +82,19 @@ jobs:
8082
max-parallel: 3
8183
matrix:
8284
provider: [github, gitlab, gitea]
85+
# One group per source (PR number, else branch)/provider, so a repeated
86+
# push or a workflow rerun cancels its own predecessor instead of piling
87+
# up runner/provider load -- but this is NOT a cleanup mechanism (see
88+
# scripts/e2e-harness.sh's per-run branch naming): a cancelled run's
89+
# branch can still be mid-delete when the next one starts, which is
90+
# exactly why every run gets its own unique branch regardless of
91+
# cancellation. `concurrency:` at job level *does* support the `matrix`
92+
# context (unlike job-level `if:`, see the comment below), so each
93+
# matrix leg gets its own group.
94+
concurrency:
95+
group: >-
96+
e2e-${{ github.event.pull_request.number && format('pr-{0}', github.event.pull_request.number) || format('branch-{0}', github.ref_name) }}-${{ matrix.provider }}
97+
cancel-in-progress: true
8398
env:
8499
E2E_GITHUB_OWNER: ${{ vars.E2E_GITHUB_OWNER }}
85100
E2E_GITHUB_REPO: ${{ vars.E2E_GITHUB_REPO }}
@@ -91,6 +106,17 @@ jobs:
91106
E2E_GITLAB_PROJECT_ID: ${{ secrets.E2E_GITLAB_PROJECT_ID }}
92107
E2E_GITLAB_TOKEN: ${{ secrets.E2E_GITLAB_TOKEN }}
93108
E2E_KEEP_BRANCH: ${{ github.event.inputs.keep_branch }}
109+
# Identity inputs for scripts/e2e-namespace.sh (via e2e-harness.sh's
110+
# `provision`): PR runs get e2e/pr/<number>/**, everything else
111+
# (push/workflow_dispatch/schedule) is a branch-only run under
112+
# e2e/branch/<id>/**. github.head_ref is only set for pull_request
113+
# events; ref_name covers push/dispatch/schedule.
114+
E2E_PR_NUMBER: ${{ github.event.pull_request.number }}
115+
E2E_SOURCE_BRANCH: ${{ github.head_ref || github.ref_name }}
116+
# Unique per run_id/run_attempt/provider so a previous killed job's
117+
# local files can never leak into this one -- never a shared/reused
118+
# directory across runs (see docs/testing/real-provider-e2e.md).
119+
E2E_WORKDIR: ${{ runner.temp }}/git-files-sync-e2e/${{ github.run_id }}/${{ github.run_attempt }}/${{ matrix.provider }}
94120
steps:
95121
# Per-provider gate (needs `matrix`, so it runs as a step, not the job-level
96122
# `if:` above -- see the comment on that `if:` for why). A fork PR (head repo
@@ -139,30 +165,25 @@ jobs:
139165
if: steps.gate.outputs.run == 'true'
140166

141167
# Arrange/Assert/cleanup are Shell + Git (scripts/e2e-harness.sh); Act
142-
# stays production TypeScript (npx vitest). $E2E_WORKDIR is fixed for
143-
# the whole job so all four steps below share the same run state/
144-
# generated runtime adapters (see scripts/e2e-harness.sh's own
145-
# `workdir` comment) -- set once here rather than depending on each
146-
# step's own default.
168+
# stays production TypeScript (npx vitest). E2E_WORKDIR/E2E_PR_NUMBER/
169+
# E2E_SOURCE_BRANCH are set once at job level (see the job `env:`
170+
# above) so all steps below share the same run state/identity.
147171
- name: Provision isolated branch/container
148172
if: steps.gate.outputs.run == 'true'
149173
env:
150174
E2E_PROVIDER: ${{ matrix.provider }}
151-
E2E_WORKDIR: ${{ runner.temp }}/e2e-${{ matrix.provider }}
152175
run: scripts/e2e-harness.sh provision
153176

154177
- name: Seed baseline fixture
155178
if: steps.gate.outputs.run == 'true'
156179
env:
157180
E2E_PROVIDER: ${{ matrix.provider }}
158-
E2E_WORKDIR: ${{ runner.temp }}/e2e-${{ matrix.provider }}
159181
run: scripts/e2e-harness.sh seed
160182

161183
- name: Run provider E2E (production TypeScript, real provider)
162184
if: steps.gate.outputs.run == 'true'
163185
env:
164186
E2E_PROVIDER: ${{ matrix.provider }}
165-
E2E_WORKDIR: ${{ runner.temp }}/e2e-${{ matrix.provider }}
166187
run: |
167188
set -a
168189
# shellcheck disable=SC1091
@@ -175,14 +196,17 @@ jobs:
175196
if: steps.gate.outputs.run == 'true'
176197
env:
177198
E2E_PROVIDER: ${{ matrix.provider }}
178-
E2E_WORKDIR: ${{ runner.temp }}/e2e-${{ matrix.provider }}
179199
run: scripts/e2e-harness.sh verify
180200

201+
# `if: always()` -- cleanup is best-effort, never a prerequisite for
202+
# the next run (see scripts/e2e-harness.sh's cmd_cleanup and
203+
# docs/testing/real-provider-e2e.md's cleanup hierarchy); a
204+
# cancelled/killed job still gets a shot at this step, but the next
205+
# run never depends on it succeeding.
181206
- name: Cleanup
182207
if: always() && steps.gate.outputs.run == 'true'
183208
env:
184209
E2E_PROVIDER: ${{ matrix.provider }}
185-
E2E_WORKDIR: ${{ runner.temp }}/e2e-${{ matrix.provider }}
186210
run: scripts/e2e-harness.sh cleanup
187211

188212
# Aggregates the matrix into a single required status so branch protection
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: E2E branch cleanup
2+
3+
# Layer 2 of the cleanup hierarchy (docs/testing/real-provider-e2e.md):
4+
# authoritative for the e2e/branch/<id>/** namespace once its source branch
5+
# is deleted. `delete` events always run from the default branch's workflow
6+
# file (the deleted ref can't supply one), so this is inherently trusted --
7+
# no `pull_request_target`-style checkout concern applies here.
8+
on:
9+
delete:
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
cleanup:
16+
name: E2E branch cleanup / ${{ matrix.provider }}
17+
if: github.event.ref_type == 'branch'
18+
runs-on: ubuntu-latest
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
# Gitea has no persistent branches to clean up -- its whole
23+
# container is torn down per-run by scripts/e2e-harness.sh cleanup.
24+
provider: [github, gitlab]
25+
# Same group naming as the provider-e2e job in ci.yml
26+
# (e2e-branch-<source-branch>-<provider>) so this cleanup queues
27+
# behind, rather than races, an E2E run still in flight for the same
28+
# branch/provider. cancel-in-progress: false -- must never cancel an
29+
# active E2E run, only run after it.
30+
concurrency:
31+
group: e2e-branch-${{ github.event.ref }}-${{ matrix.provider }}
32+
cancel-in-progress: false
33+
steps:
34+
- uses: actions/checkout@v6
35+
36+
- name: Delete e2e/branch/<id>/** namespace
37+
env:
38+
E2E_PROVIDER: ${{ matrix.provider }}
39+
# github.event.ref's exact format (full "refs/heads/foo" vs short
40+
# "foo") isn't consistently documented across event types; strip
41+
# any refs/heads/ prefix so this always matches the short branch
42+
# name ci.yml's provider-e2e job used at provision time
43+
# (github.head_ref || github.ref_name -- both already short).
44+
E2E_SOURCE_BRANCH_RAW: ${{ github.event.ref }}
45+
E2E_GITHUB_OWNER: ${{ vars.E2E_GITHUB_OWNER }}
46+
E2E_GITHUB_REPO: ${{ vars.E2E_GITHUB_REPO }}
47+
E2E_GITHUB_TOKEN: ${{ secrets.E2E_GITHUB_TOKEN }}
48+
E2E_GITLAB_PROJECT_ID: ${{ secrets.E2E_GITLAB_PROJECT_ID }}
49+
E2E_GITLAB_TOKEN: ${{ secrets.E2E_GITLAB_TOKEN }}
50+
run: |
51+
export E2E_SOURCE_BRANCH="${E2E_SOURCE_BRANCH_RAW#refs/heads/}"
52+
scripts/e2e-namespace-cleanup.sh

.github/workflows/e2e-janitor.yml

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: E2E janitor
2+
3+
# Layer 3 of the cleanup hierarchy (docs/testing/real-provider-e2e.md):
4+
# TTL-based garbage collection of e2e/** branches on the dedicated E2E
5+
# sandbox repos, independent of whether layer 1 (current-run cleanup, in
6+
# scripts/e2e-harness.sh) or layer 2 (PR-close / branch-delete cleanup, in
7+
# e2e-pr-cleanup.yml / e2e-branch-cleanup.yml) ever ran -- covers runner
8+
# crashes, forced cancellations, workflow timeouts, and cleanup-step
9+
# failures in either of the other two layers. Scheduled, not triggered by
10+
# any repo event, so it always runs from the default branch's own workflow
11+
# file/scripts.
12+
on:
13+
schedule:
14+
- cron: '0 */6 * * *'
15+
workflow_dispatch:
16+
inputs:
17+
ttl_hours:
18+
description: 'Delete e2e/** branches older than this many hours'
19+
default: '24'
20+
21+
permissions:
22+
contents: read
23+
24+
jobs:
25+
janitor:
26+
name: E2E janitor / ${{ matrix.provider }}
27+
runs-on: ubuntu-latest
28+
strategy:
29+
fail-fast: false
30+
matrix:
31+
# Gitea has no persistent branches to sweep -- its whole container
32+
# is torn down per-run by scripts/e2e-harness.sh cleanup.
33+
provider: [github, gitlab]
34+
steps:
35+
- uses: actions/checkout@v6
36+
37+
- name: Sweep stale e2e/** branches
38+
env:
39+
E2E_PROVIDER: ${{ matrix.provider }}
40+
TTL_HOURS: ${{ github.event.inputs.ttl_hours }}
41+
E2E_GITHUB_OWNER: ${{ vars.E2E_GITHUB_OWNER }}
42+
E2E_GITHUB_REPO: ${{ vars.E2E_GITHUB_REPO }}
43+
E2E_GITHUB_TOKEN: ${{ secrets.E2E_GITHUB_TOKEN }}
44+
E2E_GITLAB_PROJECT_ID: ${{ secrets.E2E_GITLAB_PROJECT_ID }}
45+
E2E_GITLAB_TOKEN: ${{ secrets.E2E_GITLAB_TOKEN }}
46+
run: |
47+
[ -n "$TTL_HOURS" ] && export E2E_JANITOR_TTL_SECONDS=$((TTL_HOURS * 3600))
48+
scripts/e2e-janitor.sh
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: E2E PR cleanup
2+
3+
# Layer 2 of the cleanup hierarchy (docs/testing/real-provider-e2e.md):
4+
# authoritative for the e2e/pr/<number>/** namespace once a PR closes,
5+
# merged or not. `pull_request_target` (not `pull_request`) so this always
6+
# runs the workflow file from the base branch -- trusted code -- with
7+
# `contents: read`+secrets available even for a fork PR, while never
8+
# checking out or executing anything from the closing PR's own branch (see
9+
# the checkout step below: no `ref:` override, so it resolves to the base
10+
# branch/trusted commit that triggered the event, never the PR head).
11+
on:
12+
pull_request_target:
13+
types: [closed]
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
cleanup:
20+
name: E2E PR cleanup / ${{ matrix.provider }}
21+
runs-on: ubuntu-latest
22+
strategy:
23+
fail-fast: false
24+
matrix:
25+
# Gitea has no persistent branches to clean up -- its whole
26+
# container is torn down per-run by scripts/e2e-harness.sh cleanup.
27+
provider: [github, gitlab]
28+
# Same group naming as the provider-e2e job in ci.yml
29+
# (e2e-pr-<number>-<provider>) so this cleanup queues behind, rather
30+
# than races, an E2E run still in flight for the same PR/provider.
31+
# cancel-in-progress: false -- this must never cancel an active E2E run;
32+
# it only needs to run after it.
33+
concurrency:
34+
group: e2e-pr-${{ github.event.pull_request.number }}-${{ matrix.provider }}
35+
cancel-in-progress: false
36+
steps:
37+
# No `ref:` -- actions/checkout defaults to the commit that triggered
38+
# this pull_request_target event, which is the base branch, not the
39+
# PR head. This is the trust boundary: only ever this repo's own
40+
# scripts run, never anything from the PR being closed.
41+
- uses: actions/checkout@v6
42+
43+
- name: Delete e2e/pr/${{ github.event.pull_request.number }}/** namespace
44+
env:
45+
E2E_PROVIDER: ${{ matrix.provider }}
46+
E2E_PR_NUMBER: ${{ github.event.pull_request.number }}
47+
E2E_GITHUB_OWNER: ${{ vars.E2E_GITHUB_OWNER }}
48+
E2E_GITHUB_REPO: ${{ vars.E2E_GITHUB_REPO }}
49+
E2E_GITHUB_TOKEN: ${{ secrets.E2E_GITHUB_TOKEN }}
50+
E2E_GITLAB_PROJECT_ID: ${{ secrets.E2E_GITLAB_PROJECT_ID }}
51+
E2E_GITLAB_TOKEN: ${{ secrets.E2E_GITLAB_TOKEN }}
52+
run: scripts/e2e-namespace-cleanup.sh

0 commit comments

Comments
 (0)