Build leaderboard/calibration MODIS LAI obs as a monthly mean#1806
Build leaderboard/calibration MODIS LAI obs as a monthly mean#1806AlexisRenchon wants to merge 2 commits into
Conversation
| (seconds_since_start(d) + seconds_since_start(d + Dates.Month(1))) / 2 for d in month_starts | ||
| ] | ||
| obs_var = ClimaAnalysis.resampled_as(obs_var; time = target_seconds) | ||
| obs_var = ClimaAnalysis.resampled_as(obs_var; time = mid_month_seconds) |
There was a problem hiding this comment.
I find this very difficult to read. If the problem is that the times are misaligned (e.g. the dates are on the 15th of the month instead of the start), why don't you use ClimaAnalysis.transform_dates! with Dates.firstdayofmonth instead? Is there a reason why we can't do that?
There was a problem hiding this comment.
Good call — the refactor now uses transform_dates, just not as a replacement.
It works for ERA5/ILAMB/inversion (L198/L300/L521) because those are monthly series that happen to be dated mid-month. MODIS isn't: the native samples run on a 30-day cadence that drifts against the calendar, so a month can hold two or none. In 2008, Jan and Mar hold two, Feb and Dec none:
2008-01-01, 2008-01-31, 2008-03-01, 2008-03-31, 2008-04-30, 2008-05-30, ...
Over 2000–2020 firstdayofmonth collapses 252 samples onto 210 labels, so it raises "Dates are not unique after transforming dates" (I ran it against the artifact). And there's no February sample to relabel — interpolation is what fills those months.
But you're right that it's the tool for the relabel, just after the resampling: each sample then sits in its own month and maps one-to-one onto the month start. That replaces the hand-rolled OutputVar reconstruction.
On readability — agreed. Month selection and midpoint are now _fully_observed_month_starts and _mid_month, so the body reads as: pick the fully observed months → resample to midpoints → relabel to starts. Output is bitwise identical to before.
b8f5c49 to
1dbd536
Compare
1dbd536 to
90488dd
Compare
`get_modis_lai_obs_var` resampled MODIS LAI onto an instantaneous month-start snapshot, but the model's `lai` diagnostic is a calendar- monthly *mean* dated at the month start. A month-mean labeled at the month start effectively represents mid-month, so the snapshot lagged it by ~half a month: a prescribed-MODIS-LAI run then disagreed with the LAI obs on the seasonal cycle (SIM high on the spring green-up, low on the autumn senescence) and reported a spurious monthly RMSE even though sim and obs share the same source. Sample the linearly-interpolated MODIS at the middle of each fully observed calendar month and label the sample at the month start. For this near-piecewise-linear product the mid-month value equals the monthly mean to < 0.002 m^2 m^-2, so the seasonal cycle now matches the diagnostic. The residual annual-mean RMSE (~0.34 m^2 m^-2, zero global bias) is regridding at sharp LAI gradients, not a data mismatch. The relabel goes through `ClimaAnalysis.transform_dates`, matching the ERA5, ILAMB and inversion loaders. It is valid only after the resampling, once each sample sits inside its own calendar month: the native samples drift against the calendar, so a month may hold two or none. This function also backs the LAI calibration target, so that target shifts by up to ~0.2 m^2 m^-2 in some cells/seasons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
90488dd to
258cf70
Compare
| The model's `lai` diagnostic is a calendar-monthly mean dated at the month start, | ||
| so it represents mid-month. Sampling MODIS at each month's midpoint reproduces | ||
| that convention (to < 0.002 m^2 m^-2 for this near-piecewise-linear product); a | ||
| month-start snapshot would instead lag the mean by ~half a month, biasing SIM | ||
| high on the spring green-up and low on the autumn senescence. |
There was a problem hiding this comment.
I find this confusing to read because I don't know what the dates of the LAI observational data. It would be easier to just list them and describe the transformations we need to do to fix them.
There was a problem hiding this comment.
Done — the docstring now lists the actual native dates (2008: 01-01, 01-31, 03-01, 03-31, 04-30, 05-30, 06-29, …, so Feb/Dec hold none and Jan/Mar hold two) and walks through the three transforms against them.
| obs_var = ClimaAnalysis.resampled_as( | ||
| obs_var; | ||
| time = seconds_since_start.(_mid_month.(month_starts)), |
There was a problem hiding this comment.
You should be able to pass Dates.DateTime for the time keyword argument. I am not sure if this simplifies things.
There was a problem hiding this comment.
resampled_as uses the time kwarg directly as the coordinate, which is stored as seconds since the reference date (no DateTime conversion), so passing DateTime would not work without ClimaAnalysis support. Kept the seconds conversion, now with a comment noting why.
There was a problem hiding this comment.
Oops. I was mistaken. I thought it was implemented for this function, but it isn't.
List the concrete native sample cadence and walk through the three transforms; note why the resample time coordinate is seconds, not DateTime. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| The model prescribes MODIS LAI by linear-in-time interpolation and reports `lai` | ||
| as a calendar-monthly mean dated at the month start. This obs reproduces that | ||
| convention so a prescribed-LAI run matches it one-to-one. |
There was a problem hiding this comment.
It feels strange to mention what the model does and the diagonstic is only valid when it is monthly mean. You can just state that this dataset is postprocessed, so that it match the convention that they are monthly mean with dates being on the first day of each month.
| 1. `_fully_observed_month_starts` keeps the calendar months whose start and next | ||
| start both lie within the native span, so the midpoint never extrapolates. | ||
| 2. `resampled_as` linearly interpolates the native samples to each month's | ||
| `_mid_month` midpoint. For this near-piecewise-linear product the midpoint | ||
| equals the monthly mean to < 0.002 m^2 m^-2; a month-start snapshot would | ||
| instead lag the mean by ~half a month, biasing SIM high on the spring | ||
| green-up and low on the autumn senescence. | ||
| 3. `transform_dates(_, firstdayofmonth)` relabels each midpoint to its month | ||
| start, aligning with the model diagnostic and the leaderboard's | ||
| `resampled_as(obs, sim)`. Step 1 leaves exactly one sample per month, so the | ||
| relabel is one-to-one; applied to the drifting native samples it would | ||
| instead collide the two-per-month months onto a single label. |
There was a problem hiding this comment.
This is verbose and seems overly complicated to what is being done. It is also easier to follow if you use the example above to show what each transformation is supposed to do.
2 participants
Now (this PR)
Before this PR
Problem
LAI is prescribed from MODIS, so the leaderboard's SIM and OBS LAI should be ~1:1 — but the MON lines were visibly offset.
Root cause (temporal convention mismatch): the model's
laidiagnostic is a calendar-monthly mean dated at the month start, which effectively represents mid-month.get_modis_lai_obs_varbuilt the obs as an instantaneous month-start snapshot, which the mean leads by ~half a month — so SIM ran high on the spring green-up and low on the autumn senescence, even though sim and obs come from the same MODIS files. It is not a mask issue.The other leaderboard variables already date their obs at the month start (
transform_dates!(firstdayofmonth)); LAI was the only snapshot.Fix
Sample the linearly-interpolated MODIS at the middle of each fully observed calendar month and label it at the month start. For this near-piecewise-linear product the mid-month value equals the true monthly mean to < 0.002 m² m⁻², while the month-start snapshot differs by 0.218 m² m⁻².
The MON lines now sit on top of each other, and global bias is −0.0055 m² m⁻² (≈0, unchanged — the fix is a seasonal-phase correction that cancels in the annual mean).
Per @ph-kev's review, the relabel goes through
ClimaAnalysis.transform_dateslike the other loaders, with the month selection and midpoint split into_fully_observed_month_startsand_mid_month. That refactor is behavior-preserving (bitwise identical), so the numbers above are unaffected.Residual RMSE (~0.34 m² m⁻²) — expected, not reducible
Model and MODIS share the 1°×1° resolution, so there is no resolution loss. But the grids are registered a half-cell apart (model at integer degrees, MODIS centers at half-integer), and the model ingests MODIS nearest-neighbor while the leaderboard resamples obs bilinear. The residual is a zero-mean dipole hugging sharp LAI boundaries (Amazon/Congo rims, forest–savanna edges, coasts): interior cells agree to ~0.08, top-quartile-gradient cells carry ~0.64,
corr(|SIM−OBS|, |∇LAI|) = 0.76. Matching the obs to nearest-neighbor makes it worse (0.56); reaching ~0 would require passing obs through the model's exact regrid chain, defeating the point of an independent obs.The same floor exists for other variables, but their RMSE is dominated by genuine physical model-vs-obs differences (ER 1.29, NEE 1.27, GPP 0.83 g m⁻² day⁻¹) — the signal the leaderboard is meant to show. LAI is the only prescribed variable, hence the only one where a non-physical floor matters.
Note on scope
get_modis_lai_obs_varalso backs the LAI calibration target, so this shifts that target by up to ~0.2 m² m⁻² in some cells/seasons. Correct direction, but flagging it for in-flight LAI calibrations.🤖 Generated with Claude Code