Skip to content

Materialize gene→taxon edges as in_taxon node properties#199

Merged
kevinschaper merged 5 commits into
masterfrom
populate-gene-node-properties
Jun 2, 2026
Merged

Materialize gene→taxon edges as in_taxon node properties#199
kevinschaper merged 5 commits into
masterfrom
populate-gene-node-properties

Conversation

@kevinschaper

Copy link
Copy Markdown
Collaborator

Summary

Phenio expresses gene taxon as an OWL existential restriction (?gene rdfs:subClassOf [owl:onProperty RO:0002162 ; owl:someValuesFrom ?taxon]), which KGX serializes as a separate edge with relation = "RO:0002162". For gene records we want taxon as a node property (matching the canonical shape from hgnc-ingest / ncbi-gene), not a separate edge.

This adds a materialize_gene_taxon step in PhenioTransform.parse() between KGX TSV emission and the Koza enrichment. It:

  • Reads PhenioTransform_edges.tsv, finds rows where relation == "RO:0002162" and subject is an identifiers.org/{hgnc,ncbigene}/ URI.
  • Builds a {gene → taxon} lookup, attaches in_taxon and in_taxon_label columns to PhenioTransform_nodes.tsv (label looked up against the NCBITaxon node rows in the same file).
  • Drops the materialized edges from the edges TSV.

phenio_node_sources.yaml learns about the two new columns; phenio_node_sources.py populates the corresponding biolink Gene class slots.

Context

Part of the broader pipeline cleanup tracked in monarch-initiative/monarch-app#1307. With this in place, kg-phenio's gene nodes carry the taxon information they've always semantically had but never surfaced as a queryable property — letting downstream consumers (and monarch-ingest specifically) treat phenio-derived gene nodes the same way as canonical hgnc-ingest / ncbi-gene records.

Dependencies

  • For HGNC: depends on Populate in_taxon and description on HGNC gene imports monarch-initiative/phenio#130, which adds the in_taxon OWL restriction to construct-hgnc.sparql. Without that PR, phenio.json carries no taxon edges for HGNC subjects so the materializer has nothing to fold for HGNC genes (NCBIGene works either way — its CONSTRUCT already emits the restriction).
  • For NCBIGene to fully work end-to-end: depends on Fix obograph_source prefix-map destruction and predicate mapping biolink/kgx#548 (TsvSource prefix-map fix). Until that's published, NCBIGene URI subjects still get filtered by BAD_PREFIXES's http/https entries before the materializer runs. After kgx#548 lands and a release is published, the URL→CURIE patches in phenio_node_sources.py:39-42 / phenio_edge_sources.py:57-58 and the http/https entries in BAD_PREFIXES become removable as a follow-up.

Test plan

  • End-to-end verified on synthetic input: KGX produces PhenioTransform_*.tsv from a tiny obojson (HGNC:5 → NCBITaxon:9606, NCBIGene:698782 → NCBITaxon:9544 via RO:0002162); materializer folds in the taxon; Koza node enrichment emits HGNC:5 | biolink:Gene | A1BG | infores:hgnc | … | NCBITaxon:9606 | Homo sapiens.
  • Full kg-phenio build against current phenio.json (will run as part of normal release pipeline). Expected: ncbigene gene nodes — currently filtered out by the asymmetric prefix handling — would now carry their taxon if the kgx fix were in place. With current kgx, the materializer is a no-op for any gene whose subject URI is http://... (waiting on kgx#548).

Phenio expresses gene taxon as an OWL existential restriction
(?gene rdfs:subClassOf [owl:onProperty RO:0002162 ; owl:someValuesFrom ?taxon]),
which KGX serializes as a separate edge with relation "RO:0002162". For
gene records, we want taxon as a node property (matching the canonical shape
from hgnc-ingest / ncbi-gene), not a separate edge.

Adds a materialize_gene_taxon helper that runs in PhenioTransform.parse()
between the KGX TSV emission and the Koza enrichment. It:
- Reads PhenioTransform_edges.tsv, finds rows where relation == "RO:0002162"
  and subject is an identifiers.org/{hgnc,ncbigene}/ URI.
- Builds a {gene → taxon} lookup, attaches in_taxon and in_taxon_label
  columns to PhenioTransform_nodes.tsv (in_taxon_label looked up against
  the NCBITaxon node rows in the same file).
- Drops the materialized edges from the edges TSV.

Adds in_taxon and in_taxon_label to phenio_node_sources.yaml columns and
node_properties, and populates the corresponding biolink Gene class slots
in phenio_node_sources.py.

End-to-end verified on synthetic input: HGNC:5 emits with category
biolink:Gene, in_taxon NCBITaxon:9606, in_taxon_label "Homo sapiens".
NCBIGene URIs still get filtered by BAD_PREFIXES until biolink/kgx#548
publishes (after that, the URL→CURIE patches in phenio_node_sources.py
and the http/https entries in BAD_PREFIXES become removable as a
follow-up — see monarch-initiative/monarch-app#1307).

For HGNC, this also depends on monarch-initiative/phenio#130 (which adds
the in_taxon OWL restriction to construct-hgnc.sparql) — without that PR,
phenio.json carries no taxon edges for HGNC subjects, so this materializer
has nothing to fold for HGNC genes (NCBIGene works either way).
Three small follow-ups surfaced when running the new gene-taxon materializer
against a real phenio.json with a kgx checkout that has biolink/kgx#548
applied (the prefix-map fix that finally lets identifiers.org/{hgnc,ncbigene}/
URIs contract to CURIEs through ObographSource):

- sources.py: add NCBIGene to EDGE_SOURCES and NODE_SOURCES with
  ("ncbi-gene", "Gene"). Previously NCBIGene URIs were always dropped by
  BAD_PREFIXES on the subject side, so the maps never had to know about
  them. With kgx contracting them now, phenio_node_sources.py's
  category_sources/infores_sources lookups would KeyError otherwise.

- phenio_node_sources.py: switch the category_sources / infores_sources
  lookups to .get() with safe defaults. The kgx fix lets several previously-
  filtered prefixes flow through (e.g. MAXO, dct, prov, GOREL, UBERON_CORE);
  unknown ones now keep whatever category kgx assigned and fall back to a
  lower-cased prefix as the infores name rather than crashing the whole
  transform.

- phenio_transform.py: the gene-taxon materializer's subject filter accepts
  both CURIE and URI form (HGNC:, NCBIGene:, http://identifiers.org/{hgnc,
  ncbigene}/) so it works whether the upstream kgx has the contraction fix
  or not. Also always emit in_taxon / in_taxon_label columns even when no
  rows have a value, since phenio_node_sources.yaml declares them and
  Koza checks the header on every run.

Verified end-to-end: PhenioTransform on the current phenio.json plus this
materializer attaches in_taxon to 753 of 769 NCBIGene nodes, with
NCBIGene:698782 (macaque MLH1) coming through as
"biolink:Gene | MLH1 | infores:ncbi-gene | NCBITaxon:9544 | Macaca mulatta".
HGNC nodes pass through cleanly but without taxon (they need the SPARQL
update from monarch-initiative/phenio#130 to ship in a phenio rebuild
before the materializer has anything to fold for them).
@kevinschaper

Copy link
Copy Markdown
Collaborator Author

Pushed a fixup commit (957c5d8) with three follow-ups surfaced when running PhenioTransform end-to-end against a real phenio.json + a kgx checkout with biolink/kgx#548 applied:

  • sources.py: add NCBIGene to EDGE_SOURCES and NODE_SOURCES. NCBIGene URIs were always being dropped by BAD_PREFIXES on the subject side before, so the maps never had to know about them.
  • phenio_node_sources.py: defensive .get() for category_sources / infores_sources. With kgx now contracting more URIs (MAXO, dct, prov, GOREL, UBERON_CORE …), unknown prefixes were KeyError-ing the whole transform.
  • phenio_transform.py: the materializer's subject filter accepts both CURIE and URI form (works pre- and post-kgx#548); always emits in_taxon/in_taxon_label columns even when no rows have a value, since the koza yaml declares them.

End-to-end verified locally: 753 of 769 NCBIGene nodes get in_taxon populated; sample NCBIGene:698782 (macaque MLH1) lands as biolink:Gene | MLH1 | infores:ncbi-gene | NCBITaxon:9544 | Macaca mulatta. HGNC nodes pass through cleanly but without taxon — that needs phenio#130 to ship in a phenio rebuild first.

A handful of obscure obo annotation IRIs contract to compound CURIEs
whose local part itself contains ":" (e.g.
"OIO:http://purl.org/dc/terms/contributor"). Now that the patched
kgx prefix manager picks the longest-IRI match, those rows reach
phenio_node_sources and crash the existing two-part unpack.
Replaces materialize_gene_taxon's pandas-based nodes/edges rewrite with
a stdlib-csv pre-pass that produces a small gene_taxon.tsv sidecar.

- phenio_transform.py: build_gene_taxon_lookup() streams edges/nodes
  with csv.DictReader (no pandas), writes {gene_id, taxon_id,
  taxon_label} rows next to the kgx TSVs.
- phenio_node_sources.py: loads the sidecar at module import (mirrors
  the static NODE_SOURCES/BAD_PREFIXES pattern) and populates in_taxon
  / in_taxon_label during its existing per-row pass.
- phenio_edge_sources.py: one-line filter to drop RO:0002162 edges
  whose subject is a gene, replacing the pandas-side edge rewrite.
- phenio_node_sources.yaml: drop in_taxon/in_taxon_label from the
  source-column declaration — they're injected at write time now.
- Drops the pandas dependency from phenio_transform (only file in
  kg_phenio that imported it; pandas wasn't a declared dep anyway).
- Adds tests/test_gene_taxon_lookup.py covering happy path, duplicate
  taxa, missing taxon node, and empty/absent inputs (5 tests).

End-to-end verified on the full phenio.json: 753/769 NCBIGene nodes
get in_taxon populated, 0 gene-taxon edges survive into the enriched
edges file (same numbers as the prior pandas implementation).
@kevinschaper
kevinschaper merged commit 2b7fa27 into master Jun 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant