Hazard Exposure notebook

[bjyberg]

Back-end

• API optimization: Since Q2 and Q3 use the same API, avoid calling it twice (share/cache results). - @jchemutt PRIORITY
• Exposure Dataset incorrect: Seems to be a mismatch between the exposure dataset being used & the hazard exposure values. - @bjyberg PRIORITY
  • Part of a wider data issue that needs addressed/re-run, however handled for now by limiting to usd15
• API scope: API should really only handle main filtering and querying, additional aggregation and formatting needs to happen in the notebook. - @jchemutt PRIORITY
• Performance: Overall performance is pretty poor. It take a long time for elements to update after change admin selections for example. It feels just as slow as if we were not using the API, so this definitely needs addressed. PRIORITY
  • Download data rather than fetch it on each query. I Imagine this is something like the persistent duckdb database with something like if table X does not exist, create table x.
  • Look at other bottlenecks, currently 3-5 minute wait per query is not good.
• API url as a constant at the top of the notebook rather than in-lined

Front-end

Advanced Controls

• Group/Commodity select
  • Remove custom (pick manually) as this is a duplicate of All
  • When a group is selected and no individual commodities are selected, what happens? If it is showing the total for all commodities in the group (i.e. the same as having all commodities selected) it should select all within the commodities selector.
  • Add select/deselect button to the commodity multi select. It should just be enabled with a flag of {selectAll: true} or something similar (look at the code/documentation for the multi-select).

Cross notebook - @jchemutt

Top Priorities (Week of March 1) PRIORITY

• Region multi-select: Selecting “Full Country” should automatically select all regions for that country.
  • Some bug preventing the linking/updates across notebook @bjyberg look into
• Sorting: Regions and Districts should be listed alphabetically. (Addressed if you use Brayden admin select)
• Glitchy Page: The page moves around and scrolls randomly on user input sometimes. (If I hover on q2 figure, the page scrolls up????? same with q4 and q1)
  • NOTE: I think this is due to a lot of complexity around the page structure. I think a lot of this can and should be simplified (see how other notebooks are implemented) with minimal additional divs and similar.
• Filter placement: Remove Geographies, Commodities, and Scenarios from Advanced Filters. Place them inside each question and keep them synchronized across questions.
• Summary tables: Summary tables should display the same data as the charts.
• Admin Select Pattern:
  • General Path:
    • admin0 is null/not selected -> show all of SSA at country level
    • admin0 is not null (is selected) and admin1 is null -> Show selected countries at admin 1 level (e.g., all Kenya counties)
    • admin0 is not null and admin 1 is not null -> Show all admin 2 data within the selected admin 1 regions
    • admin2 is selected -> data and map same as above and admin2 region is highlighted on the map
  • See example here of how admin pattern should be

Before Completion

• Dataset: Swap the dataset used for total exposure with the link provided by Brayden.
• Increase font size for plots. (code in meeting chat)
• Value formatting: Standardize plot values using millions / billions (consistent units across plots).
• Code Cleaning: Clean out dead/hanging code and harmonize structure with rest of atlas notebooks so it is more modular and maintainable.
• Styling: Colors for figures and page styling needs to be harmonized with rest of the notebooks, figures need to be cleaned (labels in correct places, margins, etc.)
• Notebook Copy: Notebook needs better copy. - @bjyberg
• Order: Question numbering needs to be harmonized
• Download buttons: These need added for each section and should use the Atlas button/styling.
• Move the Advanced filters to the end of the notebook rather than the beginning.
• First filter of the notebook (Admin regions, Scenarios, commodities) should be open by default rather than collapsed.
• QAQC Before handover, go through the notebook and identify and fix any bugs, formatting issues, etc. that you may find.

Individual Questions - @jchemutt

Q1 (upset plot)

• Upset Tooltip: Currently there is no tootip for the upset plot. PRIORITY
• Numbers are misaligned on the upset bars.
• Difference should be scenario1 - scenario 2, not right - left. (e.g.). ssp585 (2020) - ssp245 (2020))
• Remove tick marks on the axis (code in meeting chat)

Q2

• Interactions: Changing the number of commodities shown in Q2 should not trigger a new API call. PRIORITY
• Tooltips: Tooltips in Q2 overlap — adjust positioning/offset to prevent collisions. PRIORITY
• Bar chart: Bars are overlapping. These should be dodged and grouped like in this example PRIORITY
  • Meeting 05/03/2026: These bars are now stacked, they should be dodged as in the example linked above.
• In tables for quick insights, unit should be value/usd or whatever is selected, and % should say % exposed
• Sort order should be fixed (alphabetical) to avoid confusion when % vs absolute toggle is selected.

Q3

• Control: Remove “Include decreases in Δ”.
• Figure:
  • Strange things happening with bars, text, axis, etc. Bars are appearing over axis text, offset from the axis, etc.
  • Flip x and y so bars are horizontal, think of better way of utilizing white space
• Defaults:
  • Differences should be shown by default
• Figure: Flip x and y so bars are horizontal, think of better way of utilizing white space @jchemutt @bjyberg
• Inputs:
  • Add option (non-default) to allow option to free y axis (y axis allowed to vary between the different plots)

Q4 (the map & bars)

• Tooltip not title: Tooltip needs to be a tooltip, not a title element. Same as Tooltips need to be from Observable plot ({tip: true}) rather than HTML title elements for consistency #16.
• Bars for compound hazards should be dodged not stacked
• Computation error in the % exposed for the bars, it is correct elsewhere in the notebook, so something strange is happening here. For example, with default selections, wet (any) is less than wet (only) + other wet compound hazards (4.96 % vs > 11%). This should not be the case. Similar calculation is performed in "Which crops are most exposed?" and seems to be correct, so that should be a good place to look for differences.
• Left and right table column headings in quick insight should be changed to more useful scenario names (ssp585 - 2040, historic, etc.)
• x-axis needs to show on both top and bottom of plot.

Q5

• Periods: Q5 is missing one time period. PRIORITY
• Uncertainty: The Uncertainty control in Q5 is not working.
• Controls
  • Scenarios and periods should use multi select code provided by Brayden PRIORITY
  • The Uncertainty control in Q5 is not working. PRIORITY (Use the value_sd column in data)
  • Atlas Styling is not being applied correctly hazard dropdown.
  • Controls need formatting to look cleaner. Use Inputs.form from observable inputs.
  • Allow free y-axis selection here as well (Same behavior as q3)
• Facet plot as a grid (2x2 shape rather than current 4x1 shape)
• Historic Comparison:
  • Calculation method questionable @bjyberg @peetmate to think about method
  • Add difference from baseline (show absolute values) in addition to current index method @jchemutt
  • How to deal with index calculation when the values are small? (z-score normalize?)
    • No hazard in baseline, things get hotter so we have some increase in hazard (100$ in baseline $50m exposed in future how to handle)

• Bug in axis when uncertainty is selected. Things are flipped, formatting is not applied, etc.
• Bug where if y-axis free is not-selected and uncertainty is selected, the figure disappears
• Rename actual value to absolute value
• Tooltip needed/not appearing (use opbservable plot tip, not title element)
• Add explanation for dotted line
• Larger Text

[peetmate]

Update — the "Exposure Dataset incorrect / wider data issue needs re-run" item is root-caused and fixed

The exposure-value mismatch flagged here (and worked around by limiting to usd15) was three compounding bugs in the historic hazard_exposure VoP, now traced and fixed:

1. NDWS over-saturation (hazards#19) — historic NDWS frequency was saturated; de-saturated (≈29→21 stress days).
2. GLW4-2020 livestock density — the 2020 _Da rasters are animals per km² (the 2015 batch were per-pixel counts); the pipeline read them as counts → ~73× undercount that distorted the highland/tropical split. Fixed (× cell area).
3. VoP currency mislabel — livestock vop_intld15 was actually nominal USD (production × global nominal price), while crop vop_intld15 was constant international dollars (2015). So a single variable=vop_intld15 product mixed two currencies across commodities — the core reason the exposure looked wrong.

Fix + QAQC. Both crop and livestock VoP now use genuine FAOStat constant international dollars, distributed by production/head share (crop = SPAM production share, livestock = GLW head share). Validated against FAOStat national GPV (gridded country VoP ÷ FAOStat GPV, target ~1):

• Livestock: 1.00 (242/242 countries × species within 0.9–1.1)
• Crop: 0.99 (36/50; outliers are North Africa outside the SSA extent, small-island FAO gaps, and normal SPAM-vs-FAO per-country variance)

Consequence for this notebook: once we republish, variable=vop_intld15 will be correct and the "limit to usd15" workaround can be lifted (usd15 remains available too).

Scope (full-refresh, approved): supersedes the 2025-06-25 historic product — new model=historic (const-I$, de-saturated NDWS), adds the none subtype (CR-068), GLW4-2020, current GAUL/crop list. VoP is scenario-invariant, so future (model=ENSEMBLE) inherits the same corrected I$ basis (future NDWS frequency is unchanged — still Track-2).

One question that blocks our publish structure — @bjyberg: the combined model=ENSEMBLE parquet contains all scenarios including historic; we also derive a separate model=historic partition (to match the live read + supersede 2025-06-25). Does the notebook read historic only from model=historic (→ the historic rows inside model=ENSEMBLE are harmless duplication, we publish as-is), or does it aggregate model=ENSEMBLE across scenarios (→ we must strip historic from the ENSEMBLE file before upload to avoid double-counting)? Your answer determines what we ship.

Nothing published yet — holding on final QAQC + your read-contract answer. STAC/CDH cataloging tracked in AdaptationAtlas/data-management#2.

[peetmate]

Go / no-go — refreshed hazard_exposure is publish-ready

All data-side work is done and validated; we're holding for a go/no-go before writing to the live Atlas (it overwrites the 2025-06-25 vintage wholesale).

What's fixed (the "exposure dataset incorrect / needs re-run" item here): NDWS over-saturation (de-saturated historic), GLW4-2020 livestock density read as counts, and the currency mislabel — livestock vop_intld15 was nominal-USD while crop was constant-I$. Both commodities are now genuine FAOStat constant international dollars, QAQC'd against national GPV (livestock 1.00, crop 0.99). So variable=vop_intld15 will be correct and the usd15 workaround can be lifted.

Structure — matches the live read contract (please confirm):

• model=ENSEMBLE = ssp-only (ssp126/245/370/585), verified 0 historic rows — same as the current live product.
• model=historic = the baseline ensemble (mean + value_sd), served separately (as live does).
• Also adds the none subtype (CR-068 — value(none)+value(any)=total VOP, so % exposed is self-contained), GLW4-2020, current GAUL/crop list.

Ask @bjyberg — go / no-go? Publishing supersedes the live historic + refreshes future VOP to the same const-I$ basis. Blocking questions: (1) does the notebook read baseline from model=historic (not summing model=ENSEMBLE)? we've built to that. (2) any reason not to overwrite the 2025-06-25 keys now? On your go we publish (annual + jagermeyr, intld + usd) and run CR-068 verification on AGO. Nothing is published yet.