Skip to content

Coordinate validation against the CMOR tables - #64

Draft
swarnaleem wants to merge 6 commits into
ESGF:masterfrom
swarnaleem:feature/coordinate-standard
Draft

Coordinate validation against the CMOR tables#64
swarnaleem wants to merge 6 commits into
ESGF:masterfrom
swarnaleem:feature/coordinate-standard

Conversation

@swarnaleem

Copy link
Copy Markdown

Coordinate validation against the CMOR tables

This PR adds coordinate validation based on the CMOR tables. The checker already downloads the coordinate, grids and formula_terms tables but never uses them to validate coordinates. This new coordinate engine classifies each coordinate in a file, finds the matching table entry, and reports any differences (missing bounds, wrong units, wrong values, and so on).

What's included

  • plugins/coordinate_standard/: the engine as a small package (model, rules, classify, providers, matching, report). Key mechanism: a standard_name only gives the role (air_pressure is plev8/plev19/plev39 at once), so the exact table entry is pinned by comparing file values against each entry's requested list within the table tolerance.
  • checks/coordinate_checks/check_coordinates.py: the compliance-checker (COORD001). Soft findings (e.g. convertible but non-identical units) pass the check but are reported as LOW-severity advisories, and an error while checking one coordinate fails only that coordinate instead of crashing the whole run.
  • checks/coordinate_checks/classify_file.py and build_standard.py: standalone CLIs (exit code = FAIL count), plus table snapshots for CMIP6/CMIP7/CORDEX-CMIP6 so they run self-contained.
  • tests/test_cf_vocab.py: guards the cf-xarray vocabulary keys and spellings the classifier relies on, so a dependency bump that changes them fails in CI.

Design principles

  • Name vocabularies come from cf-xarray's public criteria tables; structural detection, CF detectors and all udunits logic from compliance-checker; parametric verticals from CF appendix D.
  • Instead of re-implementing checks that already exist, the new code reuses them. Monotonicity is checked by the existing [VAR005], now fed with the direction the table prescribes (stored_direction); coordinates that already have a TOML monotonicity rule are skipped via skip_direction_for, so nothing is checked twice. Missing coordinate variables are reported through the existing [VAR001]. Bounds shape and values remain with [VAR004]/[VAR012]. What the engine itself checks is: which table entry a coordinate is, and whether its
    attributes and values agree with that entry (units, positive, bounds presence, rank, valid range, prescribed values).
  • CmorTableProvider today; EsgvocProvider is a stub until esgvoc releases value-level coordinate metadata.

Try it

python checks/coordinate_checks/classify_file.py FILE.nc --prefix CMIP6
# --prefix: CMIP6 | CMIP7 | CORDEX-CMIP6; exit code = number of FAILs
python checks/coordinate_checks/build_standard.py --prefix CMIP6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant