feat(esm_catalog): PR-A1f — paleo extension + experiment classification#1513
feat(esm_catalog): PR-A1f — paleo extension + experiment classification#1513siligam wants to merge 2 commits into
Conversation
Adds src/esm_catalog/paleo.py: - add_paleo_extension(item, paleo_config, paleo_year, reference_year) — paleo:year, paleo:display, paleo:reference_year, paleo:epoch, paleo:period. Year priority: explicit param > paleo_config reference_year > extreme year (< 0 or > 9999) parsed from the item's own datetime strings. - add_experiment_type(item) — experiment_type (paleo/control/historical) for every item, from the paleo:year property or the item's datetimes; per the plan this no longer reads nml:echam:runctl:dt_start (fragile assumption that echam is always a component), and paleo:years_bp is only added when experiment_type is "paleo" instead of for every dated item. Plan-mandated cleanups vs the pr-a1a version: dead `if TYPE_CHECKING: pass` removed, uncalled detect_paleo_simulation removed. CollectionContext gets an optional paleo_config field (the general.paleo config section as a plain dict); make_item wires both functions in. Adds configs/stac-extensions/paleo/v1.0.0/schema.json (custom extension) and tests/test_esm_catalog/test_stac_paleo.py with schema validation.
| if "epoch" in paleo_config: | ||
| item.properties["paleo:epoch"] = paleo_config["epoch"] | ||
| if "period" in paleo_config: | ||
| item.properties["paleo:period"] = paleo_config["period"] |
There was a problem hiding this comment.
Same silent-null-write class as the roles thing on #1506, just spelled with if key in dict: + direct index instead of .get(key, default) — so a .get(..., [ grep won't catch it. A bare epoch: (or period:) under general.paleo in YAML is present-but-None, so "epoch" in paleo_config is True and "paleo:epoch": null gets written into the item. If the paleo schema types these as strings it's schema-invalid, and nothing in src/esm_catalog calls .validate(), so it ships unchecked (see #1506 discussion + repro: https://gist.github.com/siligam/58cad44b97c831c2f41d22d681c3ddbd).
Cheap fix: gate on truthiness instead of presence — if paleo_config.get("epoch"): / if paleo_config.get("period"):.
There was a problem hiding this comment.
Fixed in 19a5faf — gated on paleo_config.get("epoch")/.get("period") truthiness, plus a test_bare_epoch_period_do_not_write_null regression test. Full paleo suite green (26 passed).
A bare `epoch:`/`period:` under general.paleo is present-but-None, and `if "epoch" in paleo_config` let it through, writing a schema-invalid `"paleo:epoch": null` into the item. Nothing calls pystac .validate() in the pipeline, so it shipped unchecked. Gate on truthiness instead. Same silent-null-write class as the contacts `roles` case on #1506.
Part of the esm_catalog decomposition, per the PR-A1a foundation plan. Follows #1504 (A1b core), branches from
release. Fields 21–27.Adds
src/esm_catalog/paleo.py:add_paleo_extension(item, paleo_config, paleo_year, reference_year)— setspaleo:year,paleo:display,paleo:reference_year,paleo:epoch,paleo:period. Year priority: explicit param →paleo_config["reference_year"]→ extreme year (< 0 or > 9999) parsed from the item's own datetime strings. No-op for non-paleo items.add_experiment_type(item)— classifies every item aspaleo/control/historicalfrom thepaleo:yearproperty or the item's own datetimes.Plan-mandated fixes vs the old pr-a1a version:
_add_experiment_typeno longer readsnml:echam:runctl:dt_start(fragile assumption that echam is always a component, plus an ordering dependency on the namelist extension) — there's a regression test asserting the namelist property is ignored.paleo:years_bpis only added whenexperiment_type == "paleo", not for every dated item.if TYPE_CHECKING: passremoved; uncalleddetect_paleo_simulationremoved.CollectionContextgets an optionalpaleo_configfield (thegeneral.paleoconfig section as a plain dict);make_itemwires both functions in.Also adds
configs/stac-extensions/paleo/v1.0.0/schema.json(custom extension) andtests/test_esm_catalog/test_stac_paleo.py.Note: like #1511/#1512, this touches the same two spots in
item.py— whichever of the three merges last will need trivial conflict resolution (keep all import/call lines).Test plan
pytest tests/test_esm_catalog— 65 passed