fix(running-in-ci): don't let a failed status probe read as no incident - #913
fix(running-in-ci): don't let a failed status probe read as no incident#913tend-agent wants to merge 2 commits into
Conversation
The upstream-incident recipe piped a possibly-non-JSON body into jq, so a challenge page or any non-200 produced empty stdout — indistinguishable from a clean result, and read by the surrounding guidance as licence to file the workaround PR. Fetch first, parse second, and name a failed probe as unknown.
tend-agent
left a comment
There was a problem hiding this comment.
The fix is right about the mechanism — jq swallowing a non-JSON body into empty stdout is indistinguishable from a clean probe, and capturing curl's status separately is the only way to keep the pipeline's exit status from being jq's. Verified the exit codes on this runner: jq returns 5 on an HTML body, 4 on empty stdin, 0 on {"incidents":[]}.
One gap in the same failure mode: the guard keys on the HTTP status alone, so it only closes the half of the challenge that arrives as a non-2xx. The comment right above it names the general case — "an HTML challenge page instead of JSON" — and an edge interstitial served with 200 still lands in $INCIDENTS, still parse-errors to stderr, and still leaves stdout empty. That's the exact reading the PR sets out to prevent, reached by a different status code. A jq -e . validity check on the captured body closes the class rather than the one observed instance; suggestion inline.
(The endpoint answers this runner normally right now — http=200 with valid JSON — so the 405 looks edge- or window-specific, which is an argument for guarding the shape of the body rather than the code it came with.)
…TP status An edge challenge served with 200 passed `curl -f`, parse-errored in jq, and left stdout empty — the same reading as a clean probe. Add a `jq -e` validity check on the captured body so the guard closes the class rather than the one observed non-2xx instance.
The upstream-incident probe in
running-in-cifails open: when the status endpoint doesn't answer with JSON, the recipe prints nothing, and "nothing" is the same output it prints when there are genuinely no open incidents. The agent reads the empty result as clear and proceeds to file the workaround PR that the check exists to prevent.Observed live this run.
https://www.githubstatus.com/api/v2/incidents/unresolved.jsoncurrently answers a GitHub Actions runner with HTTP 405 and a 2,519-byteHuman VerificationHTML page, reproducibly (3/3 attempts). The documented recipe on that body:Empty stdout, and the paragraph right below the recipe keys off exactly that: "If the response is non-empty and the components/timing match the symptom … exit without a PR." Empty → no incident → file the PR. Egress is fine generally (
pypi.orgreturns 200 from the same shell), so this is the status endpoint's edge challenging the runner, not a sandbox block.The fix. Fetch first, parse second, and say so when the fetch fails. A bare
curl -f … | jq … || echodoes not work — the pipeline's exit status is jq's, and jq exits 0 on empty stdin, so the||branch never fires. Verified both halves against the live endpoint and against a reachable JSON endpoint as a positive control.One added sentence draws the distinction the recipe can't draw on its own: a probe that errored is
unknown, notclear, andunknownresolves the same way an open incident does — record the symptom, skip the workaround PR.Gate assessment
0 incidents, so this is the first failure. The defect isn't occurrence-counted, though: that the recipe cannot distinguish a failed fetch from a clean result is provable by reading it, and was demonstrated end-to-end above rather than inferred.list-recent-runs.shalready treats the same class of silent-empty as worth failing loud ("refusing to report an empty run list that would read as a false all-clear"); this brings the incident probe onto that footing.Evidence: https://gist.github.com/dca23a6e6a0d8cae2665944ba31676fb