Mismatch in NexGDDP hazard exposure vs raw exposure data

[bjyberg]

I am finding an issue with the NexGDDP hazard exposure data where some of the exposure values are greater than the total VOP for a crop/region.

For example when looking at Angola where hazard = any, there are 16 crops where the hazard exposure value is > total production value and the differences in value range from a few hundred to a few million (20 million in the case of sugarcane).

I'm assuming this is a result of the resampling in lines 132-139 of script "3_freq_x_exposure.r"

if(!is.null(crop_exposure_path)){
  crop_exposure<-terra::rast(crop_exposure_path)
        
  if(!is.null(resample_rast_path)){
    crop_exposure_density<-crop_exposure/terra::cellSize(crop_exposure,unit="ha")
    crop_exposure_density<-terra::resample(crop_exposure_density,data[[1]])
    crop_exposure<-crop_exposure_density*cellSize(crop_exposure_density,unit="ha")
  }
# and so on ... 
}

Given the difference in pixel size, I wonder if relative vales are better and we don't report the total exposed value when extracting to the admin boundaries. From that we can take the relative exposure and multiply that by the higher-resolution exposure data to get estimated exposure values.

A quick fix for now would be to extract the resampled crop_exposure and livestock_exposure data to the resampled admin boundaries as no-hazard, and we can calculate % exposed from that

[peetmate]

Yes — addressed by the in-flight full-refresh rebake, via two changes that map onto the diagnosis here

This is a different symptom from the currency issue on atlas_notebooks#27, but the same rebake fixes it, in the two ways suggested in this thread:

1. The resample flagged above (L132-139: density → bilinear → density). That pattern was replaced with a mass-conserving method="sum" resample (commit 6ab604c, "Mismatch in NexGDDP hazard exposure vs raw exposure data #9") across 0.4.1 / 0.4.4 / R/3. It no longer leaks/smears VOP mass across the SPAM (0.05°) ↔ base (0.25°) grid-size gap, which was inflating the extracted totals.
2. The "% exposed / no-hazard denominator" idea — implemented as the CR-068 none subtype. Per pixel none = 1 − prob(any), so value(none) + value(any) = total VOP for each (admin, crop). The exposed fraction is now self-contained in the parquet, so exposure can't exceed total VOP by construction.

On top of that, two VOP-magnitude bugs found while chasing this were also fixed: GLW4-2020 livestock rasters were per-km² density read as per-pixel counts, and livestock vop_intld15 was actually nominal-USD (crop was constant-I$) — both now genuine FAOStat constant-I$, QAQC-validated against national GPV (livestock 1.00, crop 0.99).

Status: all in code, landing in the full-refresh bake running now. I'll confirm here with the CR-068 probe numbers once it finishes — the >100% cases (AGO rice ~203%, sugarcane ~118%) should drop to ≤100%. Holding publish until that's verified.

[peetmate]

go / no-go — fix is validated and publish-ready

Following up on the fix summarized above: the full-refresh bake is complete and all gates are green (VOP QAQC vs FAOStat I$ — livestock 1.00, crop 0.99; mass-conserving resample + CR-068 none subtype in place). We're holding for a go/no-go before publishing to the live Atlas.

On publish I'll run the CR-068 probes on AGO and post the numbers here — the >100% cases (rice ~203%, sugarcane ~118%) should drop to ≤100%, closing this issue.

@bjyberg — OK to publish? (Same decision as AdaptationAtlas/atlas_notebooks#27.) Nothing published yet.