Skip to content

fix: the Hypatia gate could never fire — 2>&1 made it unconditionally vacuous - #68

Merged
hyperpolymath merged 1 commit into
mainfrom
fix/hypatia-vacuous-gate
Sep 3, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
fix/hypatia-vacuous-gate

Conversation

@hyperpolymath

Copy link
Copy Markdown
Contributor

The gate could never fire, on any input

.github/workflows/static-analysis-gate.yml ran Hypatia as:

HYPATIA_FORMAT=json "$HOME/hypatia/hypatia-cli.sh" scan . > hypatia-findings.json 2>&1
...
if [ ! -s hypatia-findings.json ] || ! jq empty hypatia-findings.json 2>/dev/null; then
  echo "[]" > hypatia-findings.json
fi

Hypatia writes findings to stdout and a one-line summary to stderr (hyperpolymath/hypatia, lib/hypatia/cli.ex). 2>&1 folds that summary into the JSON payload, so the payload is never valid JSON, so the jq guard fails on every run, so the [] fallback substitutes a falsely-clean result. CRITICAL is then always 0 and "Fail on critical findings" is always skipped.

This is a fake gate: it reports SUCCESS unconditionally, including on a repo full of critical findings. Its green told you nothing.

Two further defects in the same block:

  • HYP_EXIT was captured and discarded. Hypatia's exit 2 means the scanner itself failed; that was indistinguishable from a clean scan.
  • jq empty is not an array check. It succeeds on any valid JSON — a bare string, an object, null. The count expressions below it assume an array.

The fix

  • --exit-zero, which is Hypatia's own documented CI recipe for precisely this case ("use in CI when a downstream step gates on severity counts"): findings still go to stdout, exit status stops encoding "findings exist".
  • Drop 2>&1. The summary belongs on the log, not in the payload.
  • Fail loudly on HYP_EXIT != 0 — a scanner crash is now a red check instead of a clean bill of health.
  • jq -e 'type == "array"' instead of jq empty.

The panic-attack job above had the identical 2>&1 defect and is fixed the same way, with one deliberate asymmetry: a malformed panic-attack payload emits a ::warning, not a failure. Hypatia's exit-code contract is documented in its own source, so gating on it is justified; panic-attack is a downloaded release binary whose contract is not verified here, and blocking merges on an unverified tool's exit code manufactures reds nobody can action.

Verified, not assumed

Proven by positive control on hyperpolymath/rsr-template-repo (PR #61): with the fix in place the gate fired for the first time, reporting Hypatia found 2 critical security issue(s) — blocking merge on findings the old code had been silently discarding. The pre-fix code returns a clean [] on the same input.

If this PR goes red, that is the gate working

A red check here is a finding this repo already had — the scan is unchanged, only its interpretation is. Nothing in this diff introduces a defect; it stops one from being hidden. Please read the failure before assuming a regression: the counts are now real.

Upstream fix: hyperpolymath/rsr-template-repo#61 · Filed as hyperpolymath/rsr-template-repo#60


Measured estate context (2026-09-03)

Repairing this gate does not produce a quiet green estate. Across the 90 affected repos,
predicted from Hypatia's own rule source and validated file-for-file against a real run:

outcome repos
>= 1 critical (gate will now block) 74
clean 16

The bulk is 452 SD004 "descriptile in retired location" findings — the unfinished
.machine_readable/6a2/ migration becoming blocking for the first time — plus 23 banned-language
files. The modal repo scores exactly 9 (6 in 6a2/, 3 in .machine_readable/), i.e. uniform
template residue rather than per-repo mess.

This repo is in the predicted-clean set, which is why it is being opened first: it installs a
working gate at no merge cost. The 74 are being held pending the descriptile migration, so that
making the gate real does not silently block every merge in the estate.

Note this file repairs two blocking gates, not one: the Hypatia gate and panic-attack assail
(whose Fail on critical findings step ends in exit 1). A third 2>&1 fold survives on
panic-attack bridge triage; its downstream step is ::warning::-only with no exit, so it
degrades a warning rather than a gate, and is deliberately out of scope here.

…y vacuous

Hypatia writes findings to stdout and its one-line summary to stderr.
Redirecting stderr into the payload made every jq parse fail, so the []
fallback substituted a falsely-clean result, CRITICAL was always 0, and
the gate step was always skipped -- the check reported SUCCESS on any
input, including a repo full of critical findings.

- use --exit-zero, Hypatia's own documented CI recipe for exactly the
  case where a downstream step gates on severity counts
- drop 2>&1; the summary belongs on the log, not inside the JSON
- fail on HYP_EXIT != 0; a scanner crash (exit 2) was previously
  indistinguishable from a clean scan
- jq -e 'type == "array"' instead of jq empty, which succeeds on any
  valid JSON including a bare string, object or null

The panic-attack job had the identical defect and is fixed the same way,
except that a malformed payload there emits a ::warning rather than
failing: its exit-code contract is not verified in this repo.

Upstream: hyperpolymath/rsr-template-repo#61
@sonarqubecloud

sonarqubecloud Bot commented Sep 3, 2026

Copy link
Copy Markdown

@gitar-bot

gitar-bot Bot commented Sep 3, 2026

Copy link
Copy Markdown

Important

You are using the Gitar free plan. Upgrade to unlock code review, CI analysis, auto-apply, custom automations, and more.

Gitar

@hyperpolymath
hyperpolymath merged commit e4f3ea9 into main Sep 3, 2026
49 checks passed
@hyperpolymath
hyperpolymath deleted the fix/hypatia-vacuous-gate branch September 3, 2026 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant