feat(esm_catalog): PR-A1b-2 — GRIB + ECHAM scanners#1486
Open
siligam wants to merge 21 commits into
Open
Conversation
siligam
force-pushed
the
esm-catalog/pr-a1b1-scan-netcdf
branch
from
June 17, 2026 12:12
3cb83f3 to
e853cac
Compare
siligam
force-pushed
the
esm-catalog/pr-a1b2-grib-echam
branch
from
June 17, 2026 12:12
5c8147d to
01a4bd9
Compare
siligam
force-pushed
the
esm-catalog/pr-a1b1-scan-netcdf
branch
from
June 17, 2026 15:54
e853cac to
bdc93bd
Compare
siligam
force-pushed
the
esm-catalog/pr-a1b2-grib-echam
branch
from
June 17, 2026 15:55
01a4bd9 to
9498a0f
Compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ependency) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… unused field, tighten guard test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dd scan_all_namelists Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…he/import cleanups, use collection_title Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d+write across 3.9/3.12)
Generated with eccodes (regular_ll_pl_grib2 sample): single temperature message (paramId=130), 4x2 grid, 179 bytes. Used by GRIB scan unit tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rib deps Extracts scan/grib.py and scan/echam.py from source branch and rewires the `from esm_catalog.scan.upath import parse_uri` calls in both files to `from esm_catalog.uri import parse_uri` (the correct location in the scan-layer stack). Adds cfgrib + eccodes to setup.py extras_require and the CI install line. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_grib_scan.py: asserts format, all output keys, variable name, CF parameter mapping, datetime extraction, and detect dispatch for the committed tiny.grib2 fixture. - test_echam_scan.py: pure helper tests (no real ECHAM GRIB needed): _parse_codes_file, _extract_stream_type, _indicator_to_cf, is_echam_file (both codes-companion and filename-pattern detection). All 41 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_to_cf + fixture README Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_catalog/ Same convention fix as PR-0 — git's rename-detection didn't auto-relocate these (likely due to the binary fixture file), so doing it manually.
siligam
force-pushed
the
esm-catalog/pr-a1b1-scan-netcdf
branch
from
June 17, 2026 16:34
bdc93bd to
0b538f0
Compare
siligam
force-pushed
the
esm-catalog/pr-a1b2-grib-echam
branch
from
June 17, 2026 16:34
9498a0f to
12ad6e2
Compare
Contributor
Author
|
Rebased onto the corrected PR-0 base (#1483) — test relocation to |
siligam
force-pushed
the
esm-catalog/pr-a1b1-scan-netcdf
branch
from
June 24, 2026 13:10
0b538f0 to
9be7194
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR-A1b-2 — GRIB + ECHAM scanners
Completes the scan layer's format support (after #1485 added NetCDF). Adds the two GRIB scanners so
esm-catalog scanhandles ECHAM/GRIB output, including extension-less files.What's here (+1,201 lines)
scan/grib.py— generic GRIB scanner (cfgrib/eccodes) → STAC metadatascan/echam.py— ECHAM-specific scanner: companion.codestable parsing,indicatorOfParameterhandling, stream detectionscan/detect.py's lazy_dispatch_gribnow resolves (no code change needed) — magic-byte sniffing routes extension-less ECHAM outputNotes
cfgrib+eccodesadded to thecatalogextra and CI. Theeccodeswheel bundles the ECCODES C library (eccodeslib), so no apt step is needed in CI.scan → staconly; the A1a guard test still passes. 42 tests green on 3.9 + 3.12.grib.py/echam.pyeach carry an identical private copy of two codes-file helpers (from verbatim extraction) — a dedup is a candidate later cleanup.This is the last piece of Miguel's "scan run dirs" unit. Next: PR-A2 (DuckDB shard) = "construct shards". Plan: PR-A1b plan