Emit kg-phenio release metadata (kozahub-metadata-schema)#195
Merged
Conversation
kevinschaper
added a commit
to monarch-initiative/monarch-ingest
that referenced
this pull request
May 5, 2026
kg-phenio publishes its release-metadata.yaml to KG-Hub S3, not to a GitHub release, so the default `releases/latest/download/...` URL pattern doesn't reach it. Add an optional `metadata_url:` field on a `repo:` entry that overrides where the receipt is fetched from. Also support metadata-only entries (no `files:`) — they contribute to the build receipt but aren't transform targets. - ingests.yaml: new `kg_phenio` entry with metadata_url pointing at kg-hub.berkeleybop.io and no files. - ingest_utils.py: `get_release_metadata_sources()` returns (repo, url) tuples honoring the override; `is_metadata_only()` helper for filtering. - cli_utils.py: `download_release_metadata()` takes the (repo, url) list. `transform_one` raises a clear error on metadata-only entries; `transform_all` filters them out of its iteration. - Tests updated; live `ingest download-release-metadata` fetches 23/24 (kg-phenio's first release pending PR Knowledge-Graph-Hub/kg-phenio#195).
kevinschaper
marked this pull request as draft
May 6, 2026 23:01
kg-phenio now writes output/release-metadata.yaml describing the upstream sources consumed (phenio + the two upheno SSSOM mappings) and the artifacts produced (merged-kg.tar.gz). The receipt nests phenio's own per-ontology versions — read from phenio's phenio-upstream-versions.tsv release asset — as a Release under the phenio entry, which itself sits under kg-phenio. When phenio's manifest is published, an HP/MONDO/CHEBI/etc. version appearing in monarch-kg's aggregated build receipt traces back through kg-phenio → phenio → infores:<ontology> for full provenance. - versions.py (top-level, outside kg_phenio/ to avoid pulling kgx into the metadata script): get_source_versions() builds the nested phenio Release plus the two upheno peer leaves. - scripts/write_metadata.py: standard kozahub-metadata-schema wrapper. - pyproject.toml: kozahub-metadata-schema git dep. - Jenkinsfile: new Metadata stage between Merge and Publish; the release-metadata.yaml is included in the s3 sync alongside the KG tarball. For local development against an unreleased phenio checkout, set PHENIO_MANIFEST_PATH=../phenio/src/ontology/phenio-upstream-versions.tsv
- New `metadata` click command that wraps scripts/write_metadata.py so the release-metadata.yaml gets emitted as part of a local `python run.py` workflow, not just under Jenkins. Matches request to make everything reproducible without Jenkins. - Bump kozahub-metadata-schema dep from `branch = "main"` to `branch = "collapse-to-single-release-class"` so the writer emits the new single-Release-class shape (id/version at top level instead of source/source_version). Will revert to main once that PR merges. Verified locally end-to-end: with PHENIO_MANIFEST_PATH pointing at a locally-built phenio-upstream-versions.tsv, the writer emits the new shape with phenio nested as a Release whose own sources are the 19 upstream ontology Releases.
kevinschaper
marked this pull request as ready for review
May 7, 2026 15:12
kevinschaper
force-pushed
the
release-metadata
branch
from
May 7, 2026 15:12
115e744 to
e43e2bb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
kg-phenio now writes
output/release-metadata.yamldescribing the upstream sources consumed (phenio + the two upheno SSSOM mappings) and the artifacts produced (merged-kg.tar.gz). The receipt nests phenio's own per-ontology versions — read from phenio's `phenio-upstream-versions.tsv` release asset — as a `Release` inside the phenio entry, which itself sits under kg-phenio.The receipt shape (excerpt):
```yaml
source: kg-phenio
source_version: v2026-04-14 # phenio's tag, used as the primary upstream version
build_version: kg-phenio_v2026-04-14__
sources:
name: PHENIO
version: v2026-04-14
version_method: github_release_api
sources:
~20 ontologies from phenio-upstream-versions.tsv
artifacts:
```
When monarch-ingest's aggregator picks this up (follow-up PR), HP/MONDO/CHEBI/etc. versions in monarch-kg's receipt trace back through `kg-phenio → phenio → infores:`, full provenance preserved.
Files
Test plan
Notes