Ontology .owl/.json/.db can drift out of version alignment (GO aspect map, ncbitaxon.db)

[realmarcin]

The ontologies pipeline derives an ontology's artifacts from different source files that are fetched independently, so nothing guarantees the .owl, .json, and SemSQL .db used for one ontology are the same release. They happen to align today, but the architecture allows silent drift.

Where the split is

GO is the clearest case:

• The transform TSV (go_nodes.tsv/go_edges.tsv) is built from go.json (ONTOLOGIES_MAP["go"] = "go.json").
• The GO aspect map (MF/BP/CC categorization, ontology_utils.get_go_category_by_aspect → _load_go_namespace_map) reads go.db, which is a SemSQL build from go.owl.
• download.yaml fetches go.json (line 57) and go.owl (line 64) as two separate downloads.

So GO category correctness depends on go.json and go.owl being the same release. They are today (both 2026-05-19), but if a kg download ever straddles a GO release boundary (GO ships ~monthly), the aspect map (.db/.owl) won't match the terms in the TSV (.json) → GO terms silently fall through to the biolink:BiologicalProcess default. (ChEBI is immune: everything derives from one chebi.owl — verified release 252 across .owl/.json/.db.)

NCBITaxon has an analogous split: metatraits/lpsn/bacdive do label + lineage lookups against data/raw/ncbitaxon.db (an OAK-fetched prebuilt SemSQL DB, sqlite:obo:ncbitaxon), while the ncbitaxon transform output is built from data/raw/ncbitaxon_removed_subset.json / ncbitaxon.owl. The OAK-cached .db release is whatever OAK last downloaded and is not pinned to the .owl/transform release. Lower risk (labels/lineage are stable) but the same class of problem.

Related infra gap (same cluster)

The SemSQL .db files are not first-class build artifacts:

• chebi.db / go.db are built on demand from .owl inside the transform.
• ncit.db / bto.db / po.db are decompressed from .db.gz on demand.
• ncbitaxon.db is an OAK cache symlink.
• go.db in particular is not in download.yaml and nothing auto-builds it before the GO category step — a fresh checkout / cleaned data/raw yields a 0-byte go.db and miscategorizes all 38,263 GO terms as BiologicalProcess (observed this session). Same S3-retirement exposure as Repoint hardcoded bbop-sqlite S3 URLs to the semanticsql CDN before raw-bucket access is retired #595.

Suggested fixes

1. Version-alignment gate in the ontologies transform: assert the versionIRI of the .owl, .json, and derived .db match for each ontology before proceeding; fail loudly on mismatch.
2. Single source of truth per ontology/release: build the .db from the exact file the transform consumes (or derive both .json and .db from one downloaded .owl), so there is only one release in play.
3. Make go.db a declared/built artifact (Makefile target or on-demand build like chebi.db) so it can never be a 0-byte stub.
4. Optionally pin/record the release of the OAK-fetched ncbitaxon.db alongside the .owl.

Context: surfaced while fixing a 0-byte go.db and reasoning about .owl/.db/.tsv alignment during ontology re-transform + merge. Related: #595 (bbop-sqlite/semsql S3 retirement), #487 (oaklib shared cache).

[realmarcin]

Status update — 3 of the 4 suggested fixes are now shipped:

• Fix 1 (version-alignment gate): ✅ done. GO (go.owl↔go.json) in ontology .db infra: SemanticSQL CDN migration (#595) + go.db build & GO version gate (#604) #606; NCBITaxon (ncbitaxon.owl↔ncbitaxon.db) in ontologies: NCBITaxon .owl/.db version-alignment gate (completes #604) #607. Both compare release stamps and surface a mismatch loudly (GO defaults to raise since a mismatch silently corrupts categories; NCBITaxon defaults to warn since OAK-cache drift is common and low-risk). Overridable via KG_GO_VERSION_CHECK / KG_NCBITAXON_VERSION_CHECK.
• Fix 3 (make go.db a built artifact): ✅ done in ontology .db infra: SemanticSQL CDN migration (#595) + go.db build & GO version gate (#604) #606 — go.db now self-builds from go.owl via semsql make when missing/empty, so the 0-byte-stub miscategorization can't recur.
• Fix 4 (pin ncbitaxon.db release): ✅ done in ontologies: NCBITaxon .owl/.db version-alignment gate (completes #604) #607 — the gate pins the OAK ncbitaxon.db release to the transform's ncbitaxon.owl.
• Fix 2 (single source of truth per ontology): not done — this is a larger architectural change (derive .json and .db from one downloaded .owl). The version-alignment gates now detect the drift it would prevent, so fix 2 is an optional hardening rather than a correctness gap.

The demonstrated failure (0-byte go.db → all GO terms → BiologicalProcess) is closed off. Suggest either closing this as substantially addressed, or narrowing scope to just fix 2 (single-source-of-truth refactor) as a future improvement.