Skip to content

feat(esm_catalog): PR-A1f — paleo extension + experiment classification#1513

Open
siligam wants to merge 2 commits into
releasefrom
esm-catalog/pr-a1f-paleo-extension
Open

feat(esm_catalog): PR-A1f — paleo extension + experiment classification#1513
siligam wants to merge 2 commits into
releasefrom
esm-catalog/pr-a1f-paleo-extension

Conversation

@siligam

@siligam siligam commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Part of the esm_catalog decomposition, per the PR-A1a foundation plan. Follows #1504 (A1b core), branches from release. Fields 21–27.

Adds src/esm_catalog/paleo.py:

  • add_paleo_extension(item, paleo_config, paleo_year, reference_year) — sets paleo:year, paleo:display, paleo:reference_year, paleo:epoch, paleo:period. Year priority: explicit param → paleo_config["reference_year"] → extreme year (< 0 or > 9999) parsed from the item's own datetime strings. No-op for non-paleo items.
  • add_experiment_type(item) — classifies every item as paleo/control/historical from the paleo:year property or the item's own datetimes.

Plan-mandated fixes vs the old pr-a1a version:

  • _add_experiment_type no longer reads nml:echam:runctl:dt_start (fragile assumption that echam is always a component, plus an ordering dependency on the namelist extension) — there's a regression test asserting the namelist property is ignored.
  • paleo:years_bp is only added when experiment_type == "paleo", not for every dated item.
  • Dead if TYPE_CHECKING: pass removed; uncalled detect_paleo_simulation removed.

CollectionContext gets an optional paleo_config field (the general.paleo config section as a plain dict); make_item wires both functions in.

Also adds configs/stac-extensions/paleo/v1.0.0/schema.json (custom extension) and tests/test_esm_catalog/test_stac_paleo.py.

Note: like #1511/#1512, this touches the same two spots in item.py — whichever of the three merges last will need trivial conflict resolution (keep all import/call lines).

Test plan

  • pytest tests/test_esm_catalog — 65 passed
  • New tests cover: no-op behavior, explicit/config/deep-time-string year resolution, epoch/period, Ma/ka/BCE/CE formatting, ISO year parsing edge cases, all three experiment_type classes, years_bp only for paleo, namelist-property independence, make_item wiring, and jsonschema validation (schema also probed to reject bad enum values and non-integer years)

Adds src/esm_catalog/paleo.py:
- add_paleo_extension(item, paleo_config, paleo_year, reference_year) —
  paleo:year, paleo:display, paleo:reference_year, paleo:epoch, paleo:period.
  Year priority: explicit param > paleo_config reference_year > extreme year
  (< 0 or > 9999) parsed from the item's own datetime strings.
- add_experiment_type(item) — experiment_type (paleo/control/historical) for
  every item, from the paleo:year property or the item's datetimes; per the
  plan this no longer reads nml:echam:runctl:dt_start (fragile assumption
  that echam is always a component), and paleo:years_bp is only added when
  experiment_type is "paleo" instead of for every dated item.

Plan-mandated cleanups vs the pr-a1a version: dead `if TYPE_CHECKING: pass`
removed, uncalled detect_paleo_simulation removed.

CollectionContext gets an optional paleo_config field (the general.paleo
config section as a plain dict); make_item wires both functions in.

Adds configs/stac-extensions/paleo/v1.0.0/schema.json (custom extension)
and tests/test_esm_catalog/test_stac_paleo.py with schema validation.
Comment thread src/esm_catalog/paleo.py Outdated
Comment on lines +63 to +66
if "epoch" in paleo_config:
item.properties["paleo:epoch"] = paleo_config["epoch"]
if "period" in paleo_config:
item.properties["paleo:period"] = paleo_config["period"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same silent-null-write class as the roles thing on #1506, just spelled with if key in dict: + direct index instead of .get(key, default) — so a .get(..., [ grep won't catch it. A bare epoch: (or period:) under general.paleo in YAML is present-but-None, so "epoch" in paleo_config is True and "paleo:epoch": null gets written into the item. If the paleo schema types these as strings it's schema-invalid, and nothing in src/esm_catalog calls .validate(), so it ships unchecked (see #1506 discussion + repro: https://gist.github.com/siligam/58cad44b97c831c2f41d22d681c3ddbd).

Cheap fix: gate on truthiness instead of presence — if paleo_config.get("epoch"): / if paleo_config.get("period"):.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 19a5faf — gated on paleo_config.get("epoch")/.get("period") truthiness, plus a test_bare_epoch_period_do_not_write_null regression test. Full paleo suite green (26 passed).

A bare `epoch:`/`period:` under general.paleo is present-but-None, and
`if "epoch" in paleo_config` let it through, writing a schema-invalid
`"paleo:epoch": null` into the item. Nothing calls pystac .validate() in
the pipeline, so it shipped unchecked. Gate on truthiness instead.

Same silent-null-write class as the contacts `roles` case on #1506.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant