Skip to content

G02202 v6/G10016 v4 - #186

Merged
trey-stafford merged 78 commits into
mainfrom
support-amsr2-0802
Sep 29, 2025
Merged

G02202 v6/G10016 v4#186
trey-stafford merged 78 commits into
mainfrom
support-amsr2-0802

Conversation

@trey-stafford

@trey-stafford trey-stafford commented Jul 7, 2025

Copy link
Copy Markdown
Contributor
  • Replace use of AU_SI25 for AMSR2 brightness temperatures with
    NSIDC-0802. AU_SI25 is expected to be discontinued in September 2025.
  • Remove support for NRT F17 from NSIDC-0080, which was expected to be
    discontinued at the end of July 2025 (now Sept. 2026). The primary NRT CDR
    will be produced from NSIDC-0802. There will be no prototype CDR for now,
    although we anticipate possibly wanting to designate AMSR3 data as "prototype"
    once it is available.
  • Update CDR concentration calculation code to use a variable minimum threshold
    for sea ice concentration based on day of year for data from the AMSR2
    platform (NSIDC-0802). This implements the "Seki" method approach to aligning
    AMSR2-derived concentrations with DMSP.
  • Remove concept of "ancillary sources", which was set with --ancillary-source
    from CLI. This feature was not working as intended, and with the move to
    v2.0.0 (which includes updates to ancillary data), it does not make sense to
    backport those updates to ancillary files to previous versions.
  • Update G10016 NRT CDR product version to v4.0.
  • Update NSIDC infrastructure NFS paths to reflect new product major versions.
  • Update pm_tb_data dependency to >=v0.6. This allows utilizing earthaccess
    to fetch NSDIC-0001 data.

Comment thread seaice_ecdr/constants.py Outdated
@trey-stafford trey-stafford changed the title Support amsr2 0802 G02202 v6/G10016 v4 Aug 11, 2025
Comment thread .github/workflows/test.yml Outdated
@trey-stafford
trey-stafford changed the base branch from main to seaice_ecdrv1_g02202v5_g10016v3 August 28, 2025 20:51
0802 is currently a prototype dataset, but expected to be released in the near future.
We will no longer produce F17 data, just the AMSR2 as the main NRT CDR.
This was meant to distinguish it from the "prototype_amsr2" directory that sat
next to it. Now this is unnecessary.
We want to retain this capability for e.g., AMSR3
Retains the code to produce a prototype group, but makes it optional. This still
feels a little hacky, and might be nice to just remove the code to support the
"prototype" group altogether, but it may yet come in handy w/ e.g., AMSR3
Still needs work to really be useful, but it retains functionality to support
producing prototype data in the future
This will include both the update to use am2 for 2024+ data and the addition of
other processing changes
TODO: confirm this is the date we want to start with.
In practice, these were static, and the conversion from fraction to percent was
unnecessary. This provides groundwork for applying Seki method thresholds.
Reflects the latest filenames for the seki method outputs
Now the function sets the `time` parameter to have consistent units and
calendar. Also Removes the FillValue from coordinate variables. This ensures
that the daily outputs for both G10016 and G02202 are consistent

Fixes an observed bug where the `time` variable has:
* `units` attr that are based on the day of file creation
* `calendar` attr that shows "proleptic_gregorian".
Support for seki method thresholding in daily temporal interpolation
@trey-stafford
trey-stafford marked this pull request as ready for review September 29, 2025 16:30
Comment thread doc/operation.md Outdated
Comment thread seaice_ecdr/ancillary.py Outdated
Comment thread seaice_ecdr/cli/daily.py

_THIS_DIR = Path(__file__).parent

# TODO: the prototype platform start date should ideally be read from the

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue here to improve this: #184

Comment thread seaice_ecdr/constants.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements G02202 v6/G10016 v4 with significant changes to data sources, algorithm thresholds, and software infrastructure. The main purpose is to transition from discontinued data sources to current ones and implement improved sea ice concentration calculations using the "Seki" method for AMSR2 data.

  • Replace AU_SI25 with NSIDC-0802 for AMSR2 brightness temperatures
  • Implement variable AMSR2 concentration thresholds based on day-of-year using the "Seki" method
  • Remove support for NRT F17 and the --ancillary-source CLI option

Reviewed Changes

Copilot reviewed 57 out of 59 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
seaice_ecdr/validation.py Remove ancillary_source parameter and update type imports
seaice_ecdr/util.py Remove ancillary_source parameter from get_num_missing_pixels function
seaice_ecdr/tb_data.py Replace AU_SI25 with NSIDC-0802 and update channel mappings
seaice_ecdr/temporal_composite_daily.py Add variable threshold logic and remove ancillary_source parameter
seaice_ecdr/initial_daily_ecdr.py Update CDR concentration calculation with date/platform-based thresholds
seaice_ecdr/ancillary.py Add Seki threshold functions and remove ancillary sources concept
seaice_ecdr/constants.py Update product versions and NFS paths
Various test files Update tests to remove ancillary_source parameters and add missing function parameters
Comments suppressed due to low confidence (4)

seaice_ecdr/tests/unit/test_temporal_composite_daily.py:118

  • The test is calling temporally_composite_dataarray with incomplete parameters. The function signature requires additional parameters like one_sided_limit, still_missing_flag, and daily_climatology_mask that are not provided in this test call.
    with pytest.raises(RuntimeError, match=r"interp_range"):
        temporally_composite_dataarray(
            target_date=dt.date(2020, 1, 1),
            hemisphere=NORTH,
            platform=F17_PLATFORM,
            da=xr.DataArray(coords=(range(2), range(3), range(4))),
            interp_range=10,
            non_ocean_mask=xr.DataArray([False, False, False]),
        )

seaice_ecdr/tests/unit/test_temporal_composite_daily.py:161

  • The test is calling temporally_composite_dataarray with incomplete parameters. The function signature requires additional parameters like one_sided_limit, still_missing_flag, and daily_climatology_mask that are not provided in this test call.
    temporal_composite, _ = temporally_composite_dataarray(
        target_date=mock_date,
        hemisphere=NORTH,
        platform=F17_PLATFORM,
        da=initial_data_array,
        interp_range=0,
        non_ocean_mask=xr.full_like(initial_data_array.isel(time=0), False, dtype=bool),
    )

seaice_ecdr/tests/unit/test_temporal_composite_daily.py:279

  • The test is calling temporally_composite_dataarray with incomplete parameters. The function signature requires additional parameters like one_sided_limit, still_missing_flag, and daily_climatology_mask that are not provided in this test call.
    temporal_composite, temporal_flags = temporally_composite_dataarray(
        target_date=mock_date,
        hemisphere=NORTH,
        platform=F17_PLATFORM,
        da=input_data_array,
        interp_range=time_spread,
        non_ocean_mask=xr.full_like(input_data_array.isel(time=0), False, dtype=bool),
    )

seaice_ecdr/tests/unit/test_temporal_composite_daily.py:367

  • The test is calling temporally_composite_dataarray with incomplete parameters. The function signature requires additional parameters like still_missing_flag and daily_climatology_mask that are not provided in this test call.
    temporal_composite, temporal_flags = temporally_composite_dataarray(
        target_date=mock_date,
        hemisphere=NORTH,
        platform=F17_PLATFORM,
        da=input_data_array,
        interp_range=5,
        one_sided_limit=5,
        non_ocean_mask=xr.full_like(input_data_array.isel(time=0), False, dtype=bool),
    )

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread seaice_ecdr/tests/unit/test_monthly.py
Comment thread seaice_ecdr/tests/unit/test_monthly.py
Comment thread seaice_ecdr/tests/integration/test_monthly.py
Comment thread seaice_ecdr/tests/regression/test_daily_aggregate.py
@trey-stafford
trey-stafford merged commit d6671ac into main Sep 29, 2025
1 check passed
@trey-stafford
trey-stafford deleted the support-amsr2-0802 branch September 29, 2025 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants