fix: the Hypatia gate could never fire — 2>&1 made it unconditionally vacuous - #68
Merged
Merged
Conversation
…y vacuous Hypatia writes findings to stdout and its one-line summary to stderr. Redirecting stderr into the payload made every jq parse fail, so the [] fallback substituted a falsely-clean result, CRITICAL was always 0, and the gate step was always skipped -- the check reported SUCCESS on any input, including a repo full of critical findings. - use --exit-zero, Hypatia's own documented CI recipe for exactly the case where a downstream step gates on severity counts - drop 2>&1; the summary belongs on the log, not inside the JSON - fail on HYP_EXIT != 0; a scanner crash (exit 2) was previously indistinguishable from a clean scan - jq -e 'type == "array"' instead of jq empty, which succeeds on any valid JSON including a bare string, object or null The panic-attack job had the identical defect and is fixed the same way, except that a malformed payload there emits a ::warning rather than failing: its exit-code contract is not verified in this repo. Upstream: hyperpolymath/rsr-template-repo#61
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



The gate could never fire, on any input
.github/workflows/static-analysis-gate.ymlran Hypatia as:Hypatia writes findings to stdout and a one-line summary to stderr (
hyperpolymath/hypatia,lib/hypatia/cli.ex).2>&1folds that summary into the JSON payload, so the payload is never valid JSON, so thejqguard fails on every run, so the[]fallback substitutes a falsely-clean result.CRITICALis then always0and "Fail on critical findings" is always skipped.This is a fake gate: it reports SUCCESS unconditionally, including on a repo full of critical findings. Its green told you nothing.
Two further defects in the same block:
HYP_EXITwas captured and discarded. Hypatia's exit2means the scanner itself failed; that was indistinguishable from a clean scan.jq emptyis not an array check. It succeeds on any valid JSON — a bare string, an object,null. The count expressions below it assume an array.The fix
--exit-zero, which is Hypatia's own documented CI recipe for precisely this case ("use in CI when a downstream step gates on severity counts"): findings still go to stdout, exit status stops encoding "findings exist".2>&1. The summary belongs on the log, not in the payload.HYP_EXIT != 0— a scanner crash is now a red check instead of a clean bill of health.jq -e 'type == "array"'instead ofjq empty.The
panic-attackjob above had the identical2>&1defect and is fixed the same way, with one deliberate asymmetry: a malformed panic-attack payload emits a::warning, not a failure. Hypatia's exit-code contract is documented in its own source, so gating on it is justified; panic-attack is a downloaded release binary whose contract is not verified here, and blocking merges on an unverified tool's exit code manufactures reds nobody can action.Verified, not assumed
Proven by positive control on
hyperpolymath/rsr-template-repo(PR #61): with the fix in place the gate fired for the first time, reportingHypatia found 2 critical security issue(s) — blocking mergeon findings the old code had been silently discarding. The pre-fix code returns a clean[]on the same input.If this PR goes red, that is the gate working
A red check here is a finding this repo already had — the scan is unchanged, only its interpretation is. Nothing in this diff introduces a defect; it stops one from being hidden. Please read the failure before assuming a regression: the counts are now real.
Upstream fix: hyperpolymath/rsr-template-repo#61 · Filed as hyperpolymath/rsr-template-repo#60
Measured estate context (2026-09-03)
Repairing this gate does not produce a quiet green estate. Across the 90 affected repos,
predicted from Hypatia's own rule source and validated file-for-file against a real run:
The bulk is 452
SD004"descriptile in retired location" findings — the unfinished.machine_readable/6a2/migration becoming blocking for the first time — plus 23 banned-languagefiles. The modal repo scores exactly 9 (6 in
6a2/, 3 in.machine_readable/), i.e. uniformtemplate residue rather than per-repo mess.
This repo is in the predicted-clean set, which is why it is being opened first: it installs a
working gate at no merge cost. The 74 are being held pending the descriptile migration, so that
making the gate real does not silently block every merge in the estate.
Note this file repairs two blocking gates, not one: the Hypatia gate and
panic-attack assail(whose
Fail on critical findingsstep ends inexit 1). A third2>&1fold survives onpanic-attack bridge triage; its downstream step is::warning::-only with noexit, so itdegrades a warning rather than a gate, and is deliberately out of scope here.