Skip to content

Pattern B: hazard_exposure value('any') exceeds standalone crop-livestock_all VoP per-admin1 — reconcile (probe first, no blind re-bake) #12

Description

@peetmate

Symptom

In the published hazard_exposure parquet, value('any') (VoP exposed to any hazard) exceeds the standalone total VoP per-admin1 for several crops — impossible if both used the same VoP mass. AGO 1995-2014 historic ratios value('any') / total_VoP: rice 203.55%, sugarcane 117.9%, pearl-millet 107.9%, tobacco 105.3%, oilpalm/maize/soybean ~100.1-100.8%. Probe probe_cross_parquet_vop_drift.sh classifies it Pattern B = per-admin1 drift across all 27 crops (not uniform scaling).

NOT fixed by the 2026-05-30 rebake (na.rm=TRUE + hazard='none' layer) — those closed CR-068 (a)/(c). Pattern B is a distinct cross-product reconciliation issue, split out of CR-068.

What the code review found (no inflation bug in R/3)

Traced both VoP paths:

  • Numerator R/3_freq_x_exposure.Rrisk_x_exposure() data_ex <- data * exposure (L171), then zonal(..., fun=sum) (L518) over rasterize(touches=TRUE) zones.
  • Denominator standalone exposure R/0.4.4_process_exposure.R — same mapspam_pro_dir VoP tifs (L95), method="sum" mass-conserving resample (L162-164), same rasterize(touches=TRUE) zones (L68-74), zonal sum.

Both read the same atlas-harmonized MapSPAM VoP on base_rast grid (the data * exposure multiply would throw "extents do not match" otherwise — it doesn't), both mass-conserve. No grid/resample mismatch in our producer.

Likely cause (to confirm)

The probe denominator is domain=exposure/.../source=glw4-2020_spam2020AA/.../crop-livestock_all.parquetBrayden's harmonized product, vintage 2026-01-22, never republished. Different harmonization / boundary-vintage than our live numerator ⇒ per-admin1 VoP differs ⇒ Pattern B. Plus Luanda NaN (historic NDWS saturation, upstream AdaptationAtlas/hazards) poisons the admin0 rollup.

Plan — probe FIRST, do not blind re-bake

  1. Cheap localization probe (cglabs): for one country, per-admin1 zonal-sum the plain crop_vop raster R/3 uses (no hazard) vs crop-livestock_all per-admin1.
    • Match ⇒ numerator VoP clean; the >100% comes from the prob/any aggregation or the stale denominator → escalate denominator republish to Brayden.
    • Mismatch ⇒ zone/boundary-vintage or harmonization reconciliation → fix at producer, then targeted re-bake.
  2. Only after the probe localizes it: targeted fix + re-bake. A full bake is ~a working day — no speculative launches.

Confounds

Split out of CR-068 (climateRationale ISSUES.md).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions