ci: overhaul model-QC comment and result-file handling - #1062
Merged
Conversation
- Show "Model file and metabolic tasks" rows as running until the checks phase completes, instead of displaying the previous run's committed values. - Add QC check: flag reactions/metabolites removed since the base branch that were not moved to the deprecated identifier lists (qc_deprecation_completeness.csv). - Merge the structural-checks and model-QC-reports tables into one; link every check name to its explanation in the testResults README. - Reorganise data/testResults/README.md: per-file provenance, per-test explanations (anchors matching the comment), and a file index. - Combine the one-line result files (round-trip, YAML lint, metabolic tasks, growth) into a single qc_status.tsv via a qcStatus.py upsert helper. - Update the PR comment only after results are committed, in two phases (fast checks, then MEMOTE), so shown numbers and CSV links are always on the branch. - Store MEMOTE core-subset and full-suite scores in separate sections of memote_score.md so a routine run never overwrites a full-suite score; each section is compared only against the same section on the base branch. - Bump actions/github-script to v9 (Node 24) to clear the Node 20 deprecation. - Fix gene-essentiality README PR-number stamping (guarded on a never-true condition).
Model quality report
Each check name links to its explanation in the testResults README. Model checksDuplicate keys (model unloadable) and no growth block the merge; every other row is a non-blocking report.
MACAW and mass/charge balance
Model file and metabolic tasks
MEMOTETotal score: 63.2% (core subset) +43.0 ✅
Per-test scores
Full suite not run for this commit; comment The score above is the fast core subset. Comment Gene essentiality (Hart 2015)Not run automatically (it takes hours). Comment ❌ = a count rose vs the target branch (regression) · Full workflow run · this comment is edited as results come in |
The committed YAML model carries only ids and names, so MEMOTE scored every annotation section 0% even though the cross-references exist in the annotation tables. Add annotateModel.py, which attaches the database identifiers from metabolites.tsv / reactions.tsv / genes.tsv to an in-memory model, and call it in memoteSnapshot.py before writing the temporary SBML. - Maps only registry (identifiers.org) namespaces MEMOTE can validate; legacy-only columns (EHMN, HepatoNET1, Recon3D, HMR2, Ratcon) are skipped. - Normalises values to each namespace: Rhea loses its "RHEA:" prefix, KEGG metabolite ids split into compound/glycan/drug by prefix, genes get ensembl (from the id) plus uniprot and ncbigene. - The enriched model exists only in memory for the temporary SBML; nothing extra is committed.
…or MEMOTE Bring the canonical annotation helper code/annotateGEM.py (a port of annotateGEM.m) and its release caller code/io/increaseHumanGEMVersion.py into the repo, and use annotate_gem from memoteSnapshot.py instead of an ad-hoc helper. This attaches the TSV cross-references and SBO terms to the in-memory model before the temporary SBML that MEMOTE reads (nothing extra is committed). Extend annotateGEM's SBO assignment to everything MEMOTE checks: - metabolites get SBO:0000247 (simple chemical) and genes SBO:0000243 (gene); - boundary reactions split into exchange / demand / sink (SBO:0000627 / 0000628 / 0000632) via cobra's own classification, which MEMOTE also uses, so each reaction carries the term its check expects. Falls back to exchange-for-all if cobra cannot classify. Remove the interim code/test/annotateModel.py in favour of annotateGEM.
The MEMOTE fast subset finishes quickly, so the interim fast-checks commit and comment update are unnecessary. Run every check, commit once at the end, and post the comment from the committed files - keeping the invariant that the comment never shows numbers or CSV links that are not yet on the branch.
annotateGEM now delegates metabolite and reaction SBO assignment to the canonical raven_toolbox.annotation.add_sbo_terms (passing Human-GEM's biomass reaction name) instead of a hand-rolled version; it keeps the Human-GEM-specific TSV cross-reference merge, and still sets the gene SBO term (SBO:0000243) that add_sbo_terms does not cover. memoteSnapshot loads the model with raven_toolbox.io.read_yaml_model, like the other RAVEN-based tests. increaseHumanGEMVersion writes its exports with raven_toolbox.io.export_for_git (yml/ mat plain, xml/xlsx/txt annotated) instead of calling cobra's writers and hand-rolled txt / dependencies writers directly. Validated on the full model: SBO terms assigned to all metabolites/reactions/genes (biomass MAR13082 -> SBO:0000629), cross-references merged without clobbering existing ones, and export_for_git writes the annotated SBML.
export_for_git writes the .mat with cobra's default variable name (the model id, HumanGEM). Write the plain YAML/MATLAB exports explicitly instead - YAML via raven-toolbox, MATLAB via cobra with varname=humanGEM - and keep export_for_git for the annotated xml/xlsx/txt exports.
Revert the increaseHumanGEMVersion workaround (explicit write_yaml_model + save_matlab_model) now that raven-toolbox's export_for_git takes a varname argument. The plain yml/mat export is a single export_for_git call again, with varname='humanGEM' pinning the MATLAB struct name.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A set of related improvements to the model-QC pull-request comment and the committed result files under
data/testResults/.Changes
deprecatedReactions.tsv/deprecatedMetabolites.tsv(qc_deprecation_completeness.csv). Non-blocking report; needs the base-branch model tables, so it runs in CI only.testResultsREADME.qc_roundtrip_*.txt,qc_yamllint.txt,qc_tasks_*.txtandqc_growth.txtare replaced by a singleqc_status.tsv, updated through aqcStatus.pyupsert helper.model/Human-GEM.ymlin a normalised, ordered form and sorts themodel/*.tsvtables into a stable order. These land in a separate commit from thedata/testResults/*results commit, so the model files on the branch stay canonically formatted.memote_score.mdnow has separate Core subset and Full suite sections; each run rewrites only its own, so a routine subset run never overwrites a committed full-suite score, and deltas only ever compare like with like.actions/github-scriptbumped v7 → v9 (Node 24).Validation
Workflows and the composite action parse; the Python scripts compile;
buildReport.pyrenders correctly across the running/fast/final phases; all comment→README anchor links resolve; the MEMOTE section-preservation andqcStatusupsert were unit-checked. End-to-end CI behaviour will be exercised when this runs as a pull request.