Skip to content

fix: fail lesson verification before reporting - #4142

Draft
Jeremy1844 wants to merge 2 commits into
Adventech:stagefrom
Jeremy1844:fix/adv-2026-0074-verifier-exit-status
Draft

fix: fail lesson verification before reporting#4142
Jeremy1844 wants to merge 2 commits into
Adventech:stagefrom
Jeremy1844:fix/adv-2026-0074-verifier-exit-status

Conversation

@Jeremy1844

Copy link
Copy Markdown

Root cause

The verifier marked a run failed only after optional pull-request comment delivery. Local/non-PR execution returned before setting failure, and a comment-service error could throw before the process received a failing status. Known validation errors could therefore be reported while the command still exited successfully.

Change

Signal the validation failure before either local reporting or optional pull-request comment delivery, and add regression coverage for all three exit paths.

Evidence

  • Base: fe492faf7433053c9f65cfd4e1ccd03197e31e7e (stage)
  • Regression commit: 3ff034858b35d4a99d25c8de6aee4acb5cd43090 — 1/3 introduced tests passed
  • Fix commit: 74365599ac67a51b2e34c68521b56ed1cc3e48f1 — 3/3 passed on Node 16.20.2 and Node 24
  • Synthetic CLI proof: red prints two validation failures and exits 0; fixed head prints the same failures and exits 1
  • Syntax, diff, object-connectivity, dependency-tree, and secret checks passed
  • src, images, web, workflows, and encrypted artifact objects are byte-identical to the base

Scope and rollout

This PR changes verify.js and one synthetic regression test only. It contains no lesson, image, generated web, dependency, workflow, credential, schema, persistence, or deployment changes.

The repository currently has no lockfile, so native npm ci is unavailable; an ephemeral public-registry lock was used only to reproduce both supported test environments. Lockfile/toolchain and baseline dependency advisories remain separate remediation roots.

Before merge, run the hosted PR annotation/comment path and the approved production corpus with the authorized private parser package. Rollback the complete two-commit range; do not restore a successful exit status for known validation failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant