fix: retry transient GitHub API errors when posting the QC comment - #1067
Conversation
Model quality report
Each check name links to its explanation in the testResults README. Model checksDuplicate keys (model unloadable) and no growth block the merge; every other row is a non-blocking report.
MACAW and mass/charge balance
Model file and metabolic tasks
MEMOTETotal score: 63.2% (core subset) 0
Per-test scores
Full suite: 64.2% 0 · from the last The score above is the fast core subset. Comment Gene essentiality (Hart 2015)Not run automatically (it takes hours). Comment ❌ = a count rose vs the target branch (regression) · Full workflow run · this comment is edited as results come in |
8064fc2 to
0ca475a
Compare
The full MEMOTE run does its real work (running the suite, committing the scores) hours before it posts the Model QC comment, but the final github-script step made unguarded listComments/updateComment/createComment calls. A transient 503 from the GitHub API on that last call raised an unhandled HttpError and failed the whole run even though the results were already produced and committed. Wrap those calls in an exponential-backoff retry that only retries transient errors (429 and 5xx), so an API blip on the cosmetic comment update can no longer sink a multi-hour run.
0ca475a to
b393fc1
Compare
3a44dc1 to
66bfdbb
Compare
The full MEMOTE suite runs for ~40 minutes, during which develop can advance — a concurrent results run, or (on a re-run) this workflow's own earlier attempt, may already have pushed a "full MEMOTE result" commit. The "Update local branch before committing changes" step fetched but never integrated the remote, so it left local behind origin and the auto-commit push was rejected as non-fast-forward (which failed run 29538381748 attempt 2). Reset local to origin/<branch> before committing. memote_score.md — the only file this workflow commits — is preserved across the reset when the run actually produced a new score, so it wins over the remote while every other result file comes from the up-to-date remote. memote_result.json is untracked at the repo root, so the reset leaves it in place for the artifact upload.
Main improvements in this PR:
The full MEMOTE run does all of its real work — running the suite and committing the scores — hours before it posts the Model QC comment. The final
github-scriptstep in thepost-qc-commentcomposite action made unguardedlistComments/updateComment/createCommentcalls, so a transient503from the GitHub API on that last call raised an unhandledHttpErrorand failed the whole run even though the results were already produced and committed (this is what sank run 29538381748).This PR wraps those three API calls in an exponential-backoff
withRetryhelper that retries only transient errors (429and5xx), backing off 2s → 4s → 8s → 16s (capped at 30s, up to 5 attempts) and logging acore.warningon each retry. Genuine errors (auth, 404, etc.) still fail fast. Because the action is shared, this also protects the incrementalmodel-qcworkflow's comment updates.No model content, tests, or MEMOTE behaviour is changed — only the resilience of the cosmetic comment-posting step.
I hereby confirm that I have:
data/deprecatedIdentifiers/.developas target branch, and will be resolved with a squash-merge.mainas target branch, and will be resolved with a merge commit.Maintainers: after merge, credit new contributors by commenting
@all-contributors please add @username for <type>on this PR or the linked issue (see CONTRIBUTING).