Skip to content

fix(ci): don't fail benchmark on fork PRs when an alert fires#1464

Open
wakame1367 wants to merge 1 commit into
developmentseed:mainfrom
wakame1367:fix/benchmark-fork-pr-alert
Open

fix(ci): don't fail benchmark on fork PRs when an alert fires#1464
wakame1367 wants to merge 1 commit into
developmentseed:mainfrom
wakame1367:fix/benchmark-fork-pr-alert

Conversation

@wakame1367

@wakame1367 wakame1367 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

What

benchmark.yml sets fail-on-alert: false, so a performance alert is not meant to fail the job. But it also sets comment-on-alert: true, and pull requests from a fork only get a read-only GITHUB_TOKEN — so posting the alert comment fails and takes the whole job down:

##[warning]Performance alert! Previous value was 0.03 and current value is 0.04. It is 1.33x worse than previous exceeding a ratio threshold 1.3
error RequestError [HttpError]: Resource not accessible by integration
    at async leavePRComment (.../leavePRComment.js:49:42)
    at async handleAlert (.../write.js:263:21)
##[error]Process completed with exit code 1.

The result is that an external contributor gets a red ✗ with a confusing permissions error on a PR that didn't change any runtime code. It's easy to hit because at this timing granularity a 0.03s0.04s jitter already exceeds the 130% threshold.

I ran into this on #1463 (a docs-only change). Note it isn't every fork PR — only the ones where an alert happens to fire, which is why e.g. #1448 was fine.

Change

Enable the alert comment only when the pull request does not come from a fork:

comment-on-alert: ${{ github.event.pull_request.head.repo.fork != true }}

Behaviour is unchanged everywhere it currently works:

event ...head.repo.fork comment-on-alert note
push (main / dev / tags) null true unchanged; token is writable
PR from the same repo false true unchanged
PR from a fork true false alert is still reported as a workflow warning, job no longer fails

The benchmark itself still runs and the alert is still surfaced in the job log — only the unauthorized comment attempt is skipped.

Verification

  • pre-commit run --files .github/workflows/benchmark.yml passes (this includes the repo's zizmor --min-severity medium gate).
  • zizmor --persona pedantic reports exactly the same findings before and after this change (1 informational, 1 low, 1 medium — all pre-existing on main), so no new findings are introduced.
  • Since pull_request runs use the workflow definition from the PR head, this PR exercises the change against itself: the benchmark job on this (fork) PR logs comment-on-alert: false, confirming the expression resolves as intended. No alert happened to fire in that run, so the alert path itself wasn't hit here — but with commenting disabled, handleAlert no longer attempts the unauthorized write that was failing the job.

🤖 Generated with Claude Code

`fail-on-alert: false` is set, so a performance alert is not supposed to fail
the job. But with `comment-on-alert: true` the action tries to POST a PR
comment, and pull requests from a fork only get a read-only GITHUB_TOKEN, so
the write fails with "Resource not accessible by integration" and takes the
whole job down.

Only enable the alert comment when the pull request does not come from a fork.
Behaviour on push events and on same-repo pull requests is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant