G02202 v6/G10016 v4 - #186
Conversation
622bab2 to
a2d4331
Compare
0802 is currently a prototype dataset, but expected to be released in the near future.
We will no longer produce F17 data, just the AMSR2 as the main NRT CDR.
This was meant to distinguish it from the "prototype_amsr2" directory that sat next to it. Now this is unnecessary.
We want to retain this capability for e.g., AMSR3
Data are now published: https://nsidc.org/data/nsidc-0802/versions/1
Retains the code to produce a prototype group, but makes it optional. This still feels a little hacky, and might be nice to just remove the code to support the "prototype" group altogether, but it may yet come in handy w/ e.g., AMSR3
Still needs work to really be useful, but it retains functionality to support producing prototype data in the future
This will include both the update to use am2 for 2024+ data and the addition of other processing changes
TODO: confirm this is the date we want to start with.
In practice, these were static, and the conversion from fraction to percent was unnecessary. This provides groundwork for applying Seki method thresholds.
Ancillary thresholds variables
Reflects the latest filenames for the seki method outputs
Now the function sets the `time` parameter to have consistent units and calendar. Also Removes the FillValue from coordinate variables. This ensures that the daily outputs for both G10016 and G02202 are consistent Fixes an observed bug where the `time` variable has: * `units` attr that are based on the day of file creation * `calendar` attr that shows "proleptic_gregorian".
Fix nrt time units calendar
Support for seki method thresholding in daily temporal interpolation
|
|
||
| _THIS_DIR = Path(__file__).parent | ||
|
|
||
| # TODO: the prototype platform start date should ideally be read from the |
This ensures that the seki thresholds defined for AMSR2 are only used for AMSR2 platform.
There was a problem hiding this comment.
Pull Request Overview
This PR implements G02202 v6/G10016 v4 with significant changes to data sources, algorithm thresholds, and software infrastructure. The main purpose is to transition from discontinued data sources to current ones and implement improved sea ice concentration calculations using the "Seki" method for AMSR2 data.
- Replace
AU_SI25withNSIDC-0802for AMSR2 brightness temperatures - Implement variable AMSR2 concentration thresholds based on day-of-year using the "Seki" method
- Remove support for NRT F17 and the
--ancillary-sourceCLI option
Reviewed Changes
Copilot reviewed 57 out of 59 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| seaice_ecdr/validation.py | Remove ancillary_source parameter and update type imports |
| seaice_ecdr/util.py | Remove ancillary_source parameter from get_num_missing_pixels function |
| seaice_ecdr/tb_data.py | Replace AU_SI25 with NSIDC-0802 and update channel mappings |
| seaice_ecdr/temporal_composite_daily.py | Add variable threshold logic and remove ancillary_source parameter |
| seaice_ecdr/initial_daily_ecdr.py | Update CDR concentration calculation with date/platform-based thresholds |
| seaice_ecdr/ancillary.py | Add Seki threshold functions and remove ancillary sources concept |
| seaice_ecdr/constants.py | Update product versions and NFS paths |
| Various test files | Update tests to remove ancillary_source parameters and add missing function parameters |
Comments suppressed due to low confidence (4)
seaice_ecdr/tests/unit/test_temporal_composite_daily.py:118
- The test is calling
temporally_composite_dataarraywith incomplete parameters. The function signature requires additional parameters likeone_sided_limit,still_missing_flag, anddaily_climatology_maskthat are not provided in this test call.
with pytest.raises(RuntimeError, match=r"interp_range"):
temporally_composite_dataarray(
target_date=dt.date(2020, 1, 1),
hemisphere=NORTH,
platform=F17_PLATFORM,
da=xr.DataArray(coords=(range(2), range(3), range(4))),
interp_range=10,
non_ocean_mask=xr.DataArray([False, False, False]),
)
seaice_ecdr/tests/unit/test_temporal_composite_daily.py:161
- The test is calling
temporally_composite_dataarraywith incomplete parameters. The function signature requires additional parameters likeone_sided_limit,still_missing_flag, anddaily_climatology_maskthat are not provided in this test call.
temporal_composite, _ = temporally_composite_dataarray(
target_date=mock_date,
hemisphere=NORTH,
platform=F17_PLATFORM,
da=initial_data_array,
interp_range=0,
non_ocean_mask=xr.full_like(initial_data_array.isel(time=0), False, dtype=bool),
)
seaice_ecdr/tests/unit/test_temporal_composite_daily.py:279
- The test is calling
temporally_composite_dataarraywith incomplete parameters. The function signature requires additional parameters likeone_sided_limit,still_missing_flag, anddaily_climatology_maskthat are not provided in this test call.
temporal_composite, temporal_flags = temporally_composite_dataarray(
target_date=mock_date,
hemisphere=NORTH,
platform=F17_PLATFORM,
da=input_data_array,
interp_range=time_spread,
non_ocean_mask=xr.full_like(input_data_array.isel(time=0), False, dtype=bool),
)
seaice_ecdr/tests/unit/test_temporal_composite_daily.py:367
- The test is calling
temporally_composite_dataarraywith incomplete parameters. The function signature requires additional parameters likestill_missing_flaganddaily_climatology_maskthat are not provided in this test call.
temporal_composite, temporal_flags = temporally_composite_dataarray(
target_date=mock_date,
hemisphere=NORTH,
platform=F17_PLATFORM,
da=input_data_array,
interp_range=5,
one_sided_limit=5,
non_ocean_mask=xr.full_like(input_data_array.isel(time=0), False, dtype=bool),
)
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
AU_SI25for AMSR2 brightness temperatures withNSIDC-0802.AU_SI25is expected to be discontinued in September 2025.NSIDC-0080, which was expected to bediscontinued at the end of July 2025 (now Sept. 2026). The primary NRT CDR
will be produced from
NSIDC-0802. There will be no prototype CDR for now,although we anticipate possibly wanting to designate AMSR3 data as "prototype"
once it is available.
for sea ice concentration based on day of year for data from the AMSR2
platform (NSIDC-0802). This implements the "Seki" method approach to aligning
AMSR2-derived concentrations with DMSP.
--ancillary-sourcefrom CLI. This feature was not working as intended, and with the move to
v2.0.0 (which includes updates to ancillary data), it does not make sense to
backport those updates to ancillary files to previous versions.
pm_tb_datadependency to >=v0.6. This allows utilizingearthaccessto fetch NSDIC-0001 data.