Refresh multi-organism validation on the deterministic placement (stacks on #83) - #84
Open
edkerk wants to merge 2 commits into
Open
Refresh multi-organism validation on the deterministic placement (stacks on #83)#84edkerk wants to merge 2 commits into
edkerk wants to merge 2 commits into
Conversation
The placement master maximises a gene-localisation objective that never mentions the per-reaction placement variable, so each reaction's compartment was a free co-optimum: the pinned solver returned it reproducibly but arbitrarily, giving 52.8% reaction agreement with curated yeast-GEM (an earlier un-pinned build happened to land ~72%). Add a lexicographic second pass: fix the gene layout to the primary optimum (fixing the y solution, not the objective value, so there is no tolerance to tune), then place each reaction in the compartment its own enzymes are predicted to occupy -- the summed DeepLoc score of the reaction's genes, with a small default_compartment prior so genes-free and score-tied reactions fall there deterministically. Yeast reaction agreement rises to 72.5% (1408/1943) and now rests on the localisation evidence rather than a solver tie-break; gene agreement is unchanged (88.7%, 716/807 -- the gene layout is untouched); coherent placement adds fewer transports (1001 -> 967); growth and blocked fraction unchanged; the warm-started second solve adds ~4s. Reproducible across independent runs. Docs updated (yeast_validation, localization_redesign, multiorganism_validation footnote); the multi-organism reaction-agreement rows await re-measurement.
Re-measure Human-GEM, AraCore, and iCre1355 (scripts/benchmark_certified_multiorg.py) on the deterministic score-aligned reaction placement, and refresh the yeast row to match yeast_validation.md. The previous reaction-agreement figures came from the old arbitrary co-optimum placement. Reaction agreement rose or held (Human-GEM 52.8% -> 56.9%, the largest move; AraCore 81.8% -> 82.1%; iCre1355 53.4% -> 54.1%; yeast 72.0% -> 72.5%), and coherent placement needs fewer transports everywhere. Gene agreement is unchanged (the tie-break leaves the gene layout untouched), confirmed per model. One non-obvious shift: iCre1355's blocked fraction rose 19.0% -> 32.9% as tighter, less transport-heavy placement leaves more reactions dead-ended (still certifies). Removes the "await re-measurement" footnote; updates the prose reaction counts to the current Human-GEM version (12854 -> 12877).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the multi-organism validation for the deterministic placement tie-break in #83.
#83 fixed the reaction-placement co-optimum degeneracy but only re-measured yeast, leaving the Human-GEM / AraCore / iCre1355 reaction-agreement figures in
multiorganism_validation.mdfrom the old arbitrary placement (footnoted "await re-measurement"). This re-runsbenchmark_certified_multiorg.pyon the tie-break code and refreshes those rows.Result
Also removes the "await re-measurement" footnote and bumps the prose Human-GEM reaction count to the on-disk model version (12854 → 12877). All three organisms certified; measured with
cobra.Configuration().processes = 1for Windowsfind_blocked_reactionssafety; reproducible run-to-run.