Skip to content

normalize.py: use merged.monarch context (fixes FBbt/WBbt prefix uppercasing)#201

Merged
kevinschaper merged 1 commit into
Knowledge-Graph-Hub:masterfrom
kevinschaper:fix/normalize-use-merged-monarch
Jun 4, 2026
Merged

normalize.py: use merged.monarch context (fixes FBbt/WBbt prefix uppercasing)#201
kevinschaper merged 1 commit into
Knowledge-Graph-Hub:masterfrom
kevinschaper:fix/normalize-use-merged-monarch

Conversation

@kevinschaper

Copy link
Copy Markdown
Collaborator

Summary

Swap the clean_and_normalize_graph contexts argument from [\"obo\", \"bioregistry.upper\"] to a single [\"merged.monarch\"]. Fixes a CURIE-casing bug where OBO mixed-case anatomy prefixes (FBbt, WBbt) were being emitted as all-uppercase (FBBT, WBBT) in merged-kg_nodes.tsv.

Why the existing code produces uppercase

The two-context combination of \"obo\" + \"bioregistry.upper\" is fed to universalizer.norm.clean_and_normalize_graph, which builds reverse-IRI maps in make_id_maps:

all_reverse_contexts = {val: key for key, val in all_contexts.items()}
all_reverse_contexts_lc = {val.lower(): key for key, val in all_contexts.items()}
all_reverse_contexts.update(all_reverse_contexts_lc)
iri_converter = Converter.from_reverse_prefix_map(all_reverse_contexts)

That lowercased-reverse-IRI key, merged on top of the canonical map, plus the two-context overlap, leaves room for non-canonical CURIEs to win on round-trips for OBO prefixes that have non-default casing — specifically FBbt: and WBbt:.

Why a single "merged.monarch" context fixes it

merged.monarch is a single-context prefixmap (16,972 entries, same data as merged with a Monarch-specific label) that has one canonical IRI prefix per CURIE. With only one entry per IRI in the reverse map, the lowercased synonym can't override anything; the canonical case wins.

Verified locally — http://purl.obolibrary.org/obo/FBbt_00000001 compresses to FBbt:00000001 and http://purl.obolibrary.org/obo/WBbt_0000001 compresses to WBbt:0000001 under merged.monarch, as do all the other prefixes monarch-app exercises.

Why this matters in practice

The 2026-06-03 monarch-kg release lost 504,820 alliance gene-expression edges to the dangling pile — essentially the entire shortfall observed on alliance_gene_to_expression_edges (1.45M vs 1.9M expected). Investigation showed 99.7% are anatomy-side missing, and 100% of those (503,187) are FBbt + WBbt — exactly the two prefixes affected by this casing bug.

missing namespace dangling edges species
FBbt 397,914 Drosophila melanogaster
WBbt 105,273 C. elegans

After this PR ships and monarch-ingest picks up the next kg-phenio release, those edges resolve natively without any monarch-ingest-side workaround.

Related

The previous `contexts=["obo", "bioregistry.upper"]` pair interacted with
universalizer's `make_id_maps` logic (which builds both canonical and
lowercased reverse-IRI maps and merges them) in a way that produced
all-uppercase prefix casings for OBO mixed-case anatomies. Specifically
the Drosophila and C. elegans anatomy CURIEs were coming out as `FBBT:`
and `WBBT:` instead of the canonical `FBbt:` and `WBbt:` shown in the
PURLs themselves.

Downstream impact in the 2026-06-03 monarch-kg release: every Alliance
gene-expression edge whose object was a Drosophila or C. elegans anatomy
node went to the dangling pile — 397,914 FBbt and 105,273 WBbt edges,
503,187 total, which was essentially the entire shortfall observed on
`alliance_gene_to_expression_edges` (1.45M vs 1.9M expected).

Swap to a single Monarch-specific context, `merged.monarch`, which:

  - ships in linkml/prefixmaps alongside `merged` (Monarch-specific clone
    of the same data; same canonical case behaviour for the prefixes
    monarch consumers care about),
  - aligns kg-phenio with monarch-app's `curie_service.py` (which is
    also being switched to `merged.monarch` in a companion PR), so the
    whole Monarch stack uses one curated CURIE convention,
  - avoids the multi-context combination problem entirely — universalizer
    builds the reverse map from a single canonical entry per IRI.

Verified locally: under `merged.monarch`,
`http://purl.obolibrary.org/obo/FBbt_00000001` compresses to
`FBbt:00000001` (and `WBbt_...` to `WBbt:...`), as does every other
prefix monarch-app exercises (MONDO / GARD / Orphanet / ICD10CM / etc.).

Tracking: monarch-initiative/monarch-app#1319.
Companion PR: monarch-initiative/monarch-app#1320.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant