Skip to content

Workflow-validation skip returns conclusion: failure on modifications but success on first-add #1343

@CSamuelLents

Description

@CSamuelLents

Observed behavior

When a PR's workflow file differs from the version on main, the anti-tamper validator skips the action. But the resulting GitHub Actions conclusion differs based on whether the PR is adding or modifying the workflow file:

First-add (PR introduces the workflow file for the first time):

##[warning]Skipping action due to workflow validation: Workflow validation failed.
The workflow file must exist and have identical content to the version on the
repository's default branch. If you're seeing this on a PR when you first add a
code review workflow file to your repository, this is normal and you should
ignore this error.
Attempt 1 failed: Workflow validation failed. ...
Error is not retryable, giving up immediately
...
Action skipped due to workflow validation error. This is expected when adding
Claude Code workflows to new repositories or on PRs with workflow changes.
Exiting due to workflow validation skip

Conclusion: success. ✅

Modification (PR changes an existing workflow file):

App token exchange failed: 401 Unauthorized - Workflow validation failed.
The workflow file must exist and have identical content to the version on the
repository's default branch. If you're seeing this on a PR when you first add a
code review workflow file to your repository, this is normal and you should
ignore this error.
Attempt 1 failed: ...
Retrying in 5 seconds...
Attempt 2 of 3...
App token exchange failed: 401 Unauthorized - Workflow validation failed. ...
Retrying in 10 seconds...
Attempt 3 of 3...
##[error]Action failed with error: Workflow validation failed. ...
Operation failed after 3 attempts
##[error]Process completed with exit code 1.

Conclusion: failure. ❌

Impact

Both code paths represent the same underlying anti-tamper check, and the user-facing effect is identical: the action does not run. But the surfaced status is opposite.

On modification, the failure status:

  1. Produces a red ✗ on the PR's checks UI, suggesting something is broken when nothing is.
  2. Contradicts the action's own message ("this is normal and you should ignore this error") by emitting a hard error rather than a warning/skip.
  3. Makes every future workflow-file improvement PR (updating allowed-tools, model, max-turns, etc.) appear to fail its own Claude review, even though that failure is the expected validator skip.
  4. Adds confusion for operators rolling out the action across multiple repos — the first repo gets a green skip, every subsequent workflow tweak gets a red failure.

Suggested fix

Align the two code paths to consistently return a non-failure conclusion on validator skip. The first-add path already does this correctly (logs as warning, exits success). The modification path should follow the same shape — detect the validation failure upstream of the retry loop and skip with success, matching the first-add behavior.

Alternatively, both paths could surface as skipped (or neutral if skipped isn't available as a conclusion) so the UI clearly distinguishes "intentionally not run" from "ran and failed".

Reproduction

  1. Add claude-code-action@v1 to a repo for the first time via PR — observe success outcome with skip.
  2. Once merged, open a follow-up PR that modifies the workflow file (e.g., change --max-turns) — observe failure outcome with same underlying skip.

Both PRs hit the same anti-tamper check; only the conclusion field differs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdev-experiencep2Non-showstopper bug or popular feature request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions