Materialize gene→taxon edges as in_taxon node properties#199
Merged
Conversation
Phenio expresses gene taxon as an OWL existential restriction
(?gene rdfs:subClassOf [owl:onProperty RO:0002162 ; owl:someValuesFrom ?taxon]),
which KGX serializes as a separate edge with relation "RO:0002162". For
gene records, we want taxon as a node property (matching the canonical shape
from hgnc-ingest / ncbi-gene), not a separate edge.
Adds a materialize_gene_taxon helper that runs in PhenioTransform.parse()
between the KGX TSV emission and the Koza enrichment. It:
- Reads PhenioTransform_edges.tsv, finds rows where relation == "RO:0002162"
and subject is an identifiers.org/{hgnc,ncbigene}/ URI.
- Builds a {gene → taxon} lookup, attaches in_taxon and in_taxon_label
columns to PhenioTransform_nodes.tsv (in_taxon_label looked up against
the NCBITaxon node rows in the same file).
- Drops the materialized edges from the edges TSV.
Adds in_taxon and in_taxon_label to phenio_node_sources.yaml columns and
node_properties, and populates the corresponding biolink Gene class slots
in phenio_node_sources.py.
End-to-end verified on synthetic input: HGNC:5 emits with category
biolink:Gene, in_taxon NCBITaxon:9606, in_taxon_label "Homo sapiens".
NCBIGene URIs still get filtered by BAD_PREFIXES until biolink/kgx#548
publishes (after that, the URL→CURIE patches in phenio_node_sources.py
and the http/https entries in BAD_PREFIXES become removable as a
follow-up — see monarch-initiative/monarch-app#1307).
For HGNC, this also depends on monarch-initiative/phenio#130 (which adds
the in_taxon OWL restriction to construct-hgnc.sparql) — without that PR,
phenio.json carries no taxon edges for HGNC subjects, so this materializer
has nothing to fold for HGNC genes (NCBIGene works either way).
Three small follow-ups surfaced when running the new gene-taxon materializer against a real phenio.json with a kgx checkout that has biolink/kgx#548 applied (the prefix-map fix that finally lets identifiers.org/{hgnc,ncbigene}/ URIs contract to CURIEs through ObographSource): - sources.py: add NCBIGene to EDGE_SOURCES and NODE_SOURCES with ("ncbi-gene", "Gene"). Previously NCBIGene URIs were always dropped by BAD_PREFIXES on the subject side, so the maps never had to know about them. With kgx contracting them now, phenio_node_sources.py's category_sources/infores_sources lookups would KeyError otherwise. - phenio_node_sources.py: switch the category_sources / infores_sources lookups to .get() with safe defaults. The kgx fix lets several previously- filtered prefixes flow through (e.g. MAXO, dct, prov, GOREL, UBERON_CORE); unknown ones now keep whatever category kgx assigned and fall back to a lower-cased prefix as the infores name rather than crashing the whole transform. - phenio_transform.py: the gene-taxon materializer's subject filter accepts both CURIE and URI form (HGNC:, NCBIGene:, http://identifiers.org/{hgnc, ncbigene}/) so it works whether the upstream kgx has the contraction fix or not. Also always emit in_taxon / in_taxon_label columns even when no rows have a value, since phenio_node_sources.yaml declares them and Koza checks the header on every run. Verified end-to-end: PhenioTransform on the current phenio.json plus this materializer attaches in_taxon to 753 of 769 NCBIGene nodes, with NCBIGene:698782 (macaque MLH1) coming through as "biolink:Gene | MLH1 | infores:ncbi-gene | NCBITaxon:9544 | Macaca mulatta". HGNC nodes pass through cleanly but without taxon (they need the SPARQL update from monarch-initiative/phenio#130 to ship in a phenio rebuild before the materializer has anything to fold for them).
Collaborator
Author
|
Pushed a fixup commit (957c5d8) with three follow-ups surfaced when running PhenioTransform end-to-end against a real phenio.json + a kgx checkout with biolink/kgx#548 applied:
End-to-end verified locally: 753 of 769 NCBIGene nodes get |
8 tasks
A handful of obscure obo annotation IRIs contract to compound CURIEs whose local part itself contains ":" (e.g. "OIO:http://purl.org/dc/terms/contributor"). Now that the patched kgx prefix manager picks the longest-IRI match, those rows reach phenio_node_sources and crash the existing two-part unpack.
Replaces materialize_gene_taxon's pandas-based nodes/edges rewrite with
a stdlib-csv pre-pass that produces a small gene_taxon.tsv sidecar.
- phenio_transform.py: build_gene_taxon_lookup() streams edges/nodes
with csv.DictReader (no pandas), writes {gene_id, taxon_id,
taxon_label} rows next to the kgx TSVs.
- phenio_node_sources.py: loads the sidecar at module import (mirrors
the static NODE_SOURCES/BAD_PREFIXES pattern) and populates in_taxon
/ in_taxon_label during its existing per-row pass.
- phenio_edge_sources.py: one-line filter to drop RO:0002162 edges
whose subject is a gene, replacing the pandas-side edge rewrite.
- phenio_node_sources.yaml: drop in_taxon/in_taxon_label from the
source-column declaration — they're injected at write time now.
- Drops the pandas dependency from phenio_transform (only file in
kg_phenio that imported it; pandas wasn't a declared dep anyway).
- Adds tests/test_gene_taxon_lookup.py covering happy path, duplicate
taxa, missing taxon node, and empty/absent inputs (5 tests).
End-to-end verified on the full phenio.json: 753/769 NCBIGene nodes
get in_taxon populated, 0 gene-taxon edges survive into the enriched
edges file (same numbers as the prior pandas implementation).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phenio expresses gene taxon as an OWL existential restriction (
?gene rdfs:subClassOf [owl:onProperty RO:0002162 ; owl:someValuesFrom ?taxon]), which KGX serializes as a separate edge withrelation = "RO:0002162". For gene records we want taxon as a node property (matching the canonical shape from hgnc-ingest / ncbi-gene), not a separate edge.This adds a
materialize_gene_taxonstep inPhenioTransform.parse()between KGX TSV emission and the Koza enrichment. It:PhenioTransform_edges.tsv, finds rows whererelation == "RO:0002162"and subject is anidentifiers.org/{hgnc,ncbigene}/URI.{gene → taxon}lookup, attachesin_taxonandin_taxon_labelcolumns toPhenioTransform_nodes.tsv(label looked up against the NCBITaxon node rows in the same file).phenio_node_sources.yamllearns about the two new columns;phenio_node_sources.pypopulates the corresponding biolink Gene class slots.Context
Part of the broader pipeline cleanup tracked in monarch-initiative/monarch-app#1307. With this in place, kg-phenio's gene nodes carry the taxon information they've always semantically had but never surfaced as a queryable property — letting downstream consumers (and monarch-ingest specifically) treat phenio-derived gene nodes the same way as canonical hgnc-ingest / ncbi-gene records.
Dependencies
in_taxonOWL restriction toconstruct-hgnc.sparql. Without that PR, phenio.json carries no taxon edges for HGNC subjects so the materializer has nothing to fold for HGNC genes (NCBIGene works either way — its CONSTRUCT already emits the restriction).BAD_PREFIXES'shttp/httpsentries before the materializer runs. After kgx#548 lands and a release is published, the URL→CURIE patches inphenio_node_sources.py:39-42/phenio_edge_sources.py:57-58and thehttp/httpsentries inBAD_PREFIXESbecome removable as a follow-up.Test plan
PhenioTransform_*.tsvfrom a tiny obojson (HGNC:5 → NCBITaxon:9606, NCBIGene:698782 → NCBITaxon:9544 viaRO:0002162); materializer folds in the taxon; Koza node enrichment emitsHGNC:5 | biolink:Gene | A1BG | infores:hgnc | … | NCBITaxon:9606 | Homo sapiens.http://...(waiting on kgx#548).