Skip to content

Commit 65326cd

Browse files
hatayamahatayama
authored
fix: make historical release recovery actionable (#1799)
1 parent 2c73c6a commit 65326cd

2 files changed

Lines changed: 165 additions & 2 deletions

File tree

‎.github/workflows/native-cli-publish.yml‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,7 @@ jobs:
282282
env:
283283
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
284284
GITHUB_REPOSITORY: ${{ github.repository }}
285+
RECOVERY_TARGET: ${{ github.event_name == 'workflow_dispatch' && inputs.recovery-target }}
285286
run: |
286287
set -eu
287288
tag_error_path="${RUNNER_TEMP}/native-cli-release-tag-error"
@@ -305,7 +306,22 @@ jobs:
305306
# Why the tag is created before the release: with an active tag ruleset the
306307
# workflow token cannot create a release for a not-yet-existing tag, while
307308
# explicit ref creation is permitted and pins the tag to the verified commit.
308-
gh api "repos/${GITHUB_REPOSITORY}/git/refs" -f ref="refs/tags/${RELEASE_TAG}" -f sha="${RELEASE_SHA}"
309+
tag_create_error_path="${RUNNER_TEMP}/native-cli-release-tag-create-error"
310+
set +e
311+
gh api "repos/${GITHUB_REPOSITORY}/git/refs" \
312+
-f ref="refs/tags/${RELEASE_TAG}" \
313+
-f sha="${RELEASE_SHA}" > "${tag_create_error_path}" 2>&1
314+
tag_create_status=$?
315+
set -e
316+
if [ "${tag_create_status}" -ne 0 ]; then
317+
cat "${tag_create_error_path}" >&2
318+
if [ "${RECOVERY_TARGET}" = "true" ] && grep -qE 'HTTP 403|Resource not accessible by integration' "${tag_create_error_path}"; then
319+
echo "Recovery-target release creation was rejected for a historical commit by GitHub." >&2
320+
echo "Create the tag and draft release as the repository owner, then rerun this workflow." >&2
321+
echo "See docs/release-recovery-runbook.md for the recovery procedure." >&2
322+
fi
323+
exit "${tag_create_status}"
324+
fi
309325
fi
310326
if gh release view "${RELEASE_TAG}" --json isDraft,targetCommitish > release-input/release-state.json 2>/dev/null; then
311327
is_draft=$(jq -r '.isDraft' release-input/release-state.json)
@@ -318,7 +334,20 @@ jobs:
318334
fi
319335
prerelease_flag=""
320336
case "${RELEASE_TAG}" in *-beta.*) prerelease_flag="--prerelease" ;; esac
321-
gh release create "${RELEASE_TAG}" --draft --title "${RELEASE_TAG}" --notes-file release-input/release-notes.md --target "${RELEASE_SHA}" ${prerelease_flag}
337+
release_create_error_path="${RUNNER_TEMP}/native-cli-release-create-error"
338+
set +e
339+
gh release create "${RELEASE_TAG}" --draft --title "${RELEASE_TAG}" --notes-file release-input/release-notes.md --target "${RELEASE_SHA}" ${prerelease_flag} > "${release_create_error_path}" 2>&1
340+
release_create_status=$?
341+
set -e
342+
if [ "${release_create_status}" -ne 0 ]; then
343+
cat "${release_create_error_path}" >&2
344+
if [ "${RECOVERY_TARGET}" = "true" ] && grep -qE 'HTTP 403|Resource not accessible by integration' "${release_create_error_path}"; then
345+
echo "Recovery-target draft release creation was rejected for a historical commit by GitHub." >&2
346+
echo "Create the tag and draft release as the repository owner, then rerun this workflow." >&2
347+
echo "See docs/release-recovery-runbook.md for the recovery procedure." >&2
348+
fi
349+
exit "${release_create_status}"
350+
fi
322351
echo "release_published=false" >> "$GITHUB_OUTPUT"
323352
324353
- name: Attest native CLI release assets

‎docs/release-recovery-runbook.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# Release recovery runbook
2+
3+
## Root cause
4+
5+
In `recovery-target` mode, the release target is an earlier commit than the
6+
workflow run's `GITHUB_SHA`. GitHub rejects the Actions `GITHUB_TOKEN` when it
7+
tries to create a tag or release for that different commit, even when the token
8+
has `contents: write`. This is neither a missing permission nor a tag ruleset
9+
failure.
10+
11+
The behavior was confirmed with the same token and `contents: write`
12+
permission in reproduction runs [29459199798](https://github.com/hatayama/unity-cli-loop/actions/runs/29459199798)
13+
and [29459266044](https://github.com/hatayama/unity-cli-loop/actions/runs/29459266044):
14+
15+
- A tag pointing at the run head returned HTTP 201.
16+
- A tag pointing at `2d8d1b9443843f45b48479134f846b6f540e928b`, an ancestor of
17+
the run head, returned HTTP 403 `Resource not accessible by integration`.
18+
- Creating a draft release directly at that past commit also returned HTTP 403.
19+
20+
The precise condition is a ref target different from the run's `GITHUB_SHA`,
21+
not merely an unreachable commit. Normal head-based releases are unaffected.
22+
23+
## Recovery procedure
24+
25+
Run these commands with an owner-authenticated `gh` session. Replace every
26+
`<placeholder>` before running a command.
27+
28+
1. Confirm the authenticated account and repository.
29+
30+
```sh
31+
gh auth status
32+
gh api user --jq .login
33+
gh api repos/<owner>/<repo> --jq .full_name
34+
```
35+
36+
2. Determine the release commit and release tag from the release PR and the
37+
failed workflow run.
38+
39+
```sh
40+
gh pr view <release-pr> --repo <owner>/<repo> --json mergeCommit --jq .mergeCommit.oid
41+
gh run view <run-id> --repo <owner>/<repo> --json headSha
42+
```
43+
44+
The release commit must be the commit validated by the workflow. Do not use
45+
the current branch head when recovering an older release.
46+
47+
3. Create the release tag at the validated release commit.
48+
49+
```sh
50+
gh api repos/<owner>/<repo>/git/refs \
51+
-f ref=refs/tags/<tag> \
52+
-f sha=<release-commit-sha>
53+
```
54+
55+
4. Download the native release input artifact and create the draft release.
56+
Creating only the tag is not sufficient: the Actions token also receives
57+
HTTP 403 when it tries to create the draft release.
58+
59+
```sh
60+
gh run download <run-id> --repo <owner>/<repo> \
61+
-n native-cli-release-input -D <native-release-input-dir>
62+
gh release create <tag> --repo <owner>/<repo> \
63+
--draft --title <tag> \
64+
--notes-file <native-release-input-dir>/release-input/release-notes.md \
65+
--target <release-commit-sha>
66+
```
67+
68+
Add `--prerelease` when the release tag is a prerelease tag.
69+
70+
5. Rerun the failed publish job and approve the release environment if
71+
prompted.
72+
73+
```sh
74+
gh run rerun <run-id> --repo <owner>/<repo> --failed
75+
```
76+
77+
6. If an older `dispatcher-publish` run is waiting for the `cli-release`
78+
environment, cancel it before retrying the recovery. The workflow uses a
79+
concurrency group without automatic cancellation, so an old approval-waiting
80+
run can block the newer run indefinitely.
81+
82+
```sh
83+
gh run list --repo <owner>/<repo> --workflow dispatcher-publish.yml \
84+
--branch <branch> --limit 20
85+
gh run cancel <blocked-run-id> --repo <owner>/<repo>
86+
```
87+
88+
7. Recover the Unity package release only after the runner release has
89+
completed and its assets are available. The package sync script creates a
90+
release at the historical release commit and can hit the same 403; create
91+
that release as the owner before rerunning the sync.
92+
93+
```sh
94+
gh release create v<package-version> --repo <owner>/<repo> \
95+
--title v<package-version> \
96+
--notes-file <package-release-notes> \
97+
--target <package-release-commit-sha>
98+
```
99+
100+
Add `--prerelease` when the package release is a prerelease.
101+
102+
Run the package sync or the relevant post-publish job after the runner
103+
release assets are complete. The sync reuses an existing correctly targeted
104+
release.
105+
106+
8. Complete the workflow-dispatch recovery manually. Push-only post-publish
107+
steps are intentionally skipped for a `workflow_dispatch` run.
108+
109+
```sh
110+
gh pr edit <release-pr> --repo <owner>/<repo> \
111+
--remove-label 'autorelease: pending' \
112+
--add-label 'autorelease: tagged'
113+
gh workflow run release-please.yml --repo <owner>/<repo> --ref <branch> \
114+
-f branch=<branch>
115+
```
116+
117+
## Scope of the workflows
118+
119+
`native-cli-publish.yml` is the workflow with `recovery-target` and now emits
120+
this runbook path when tag or draft-release creation returns the observed 403.
121+
The creation and reuse logic is unchanged.
122+
123+
`dispatcher-publish.yml` has no recovery-target input and requires its release
124+
target to equal `GITHUB_SHA`, so it has no historical-commit creation path.
125+
Its concurrency group still matters during recovery because an older run can
126+
block a retry.
127+
128+
`scripts/sync-release-please-package-releases.sh` can create package releases
129+
at release-please commits from the repository history. This runbook's owner
130+
pre-creation step covers that path; the runner release must be recovered first.
131+
132+
Do not add GitHub Actions to the tag-ruleset bypass actors. That would grant
133+
the bot tag deletion and force-update capabilities without addressing the
134+
historical-commit restriction.

0 commit comments

Comments
 (0)