Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Artifacts.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,19 @@ git-tree-sha1 = "d400472ea191639cb1dafac41b09ce36383aee54"
[[inversion_nee.download]]
sha256 = "84ccc2b60b276575342886535d78faa83f64dd83e327f65b31a10da8f9da721c"
url = "https://caltech.box.com/shared/static/13xeg76obc5cc73h2sqzhogx0hr97poz.gz"

[era5_monthly_averages_atmos_single_level_1979_2024]
git-tree-sha1 = "a3169bc6de42f7e3f4ca685654ae240dfbf560e3"
lazy = true

[[era5_monthly_averages_atmos_single_level_1979_2024.download]]
sha256 = "abf70d77ba59f5f8b5b48600ed6a8be9b819ec18fac086c0abc9c886211320c8"
url = "https://caltech.box.com/shared/static/8mo7azrz0gukdtmo2oi0dtvdt1v3t3tv.gz"

[precipitation_obs]
git-tree-sha1 = "5f0075f042bc83b580d2c4f725f851d8e0b9c20a"
lazy = true

[[precipitation_obs.download]]
sha256 = "1aea361ee1c2636bd5c4cc343202dab814d246a7e0356a9caac9fd9c7cf3821c"
url = "https://caltech.box.com/shared/static/j63uak557kw7tlzb3dgwifoaeczekzm4.gz"
13 changes: 13 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@

### New Features

#### Atmos monthly benchmarks: `pr`, `evspsbl`, `prw`
- New built-in observation loaders: GPCP monthly precipitation for `pr`
(new `precipitation_obs` artifact, ~20 MB), ERA5 mean evaporation rate for
`evspsbl` (from the already-shipped surface single-level artifact), and
ERA5 total column water for `prw` (new
`era5_monthly_averages_atmos_single_level` artifact, ~137 MB)
- Water mass fluxes (`kg m^-2 s^-1`) are now displayed in `mm day^-1`,
mirroring the carbon-flux display conversion; the negative-downward
ClimaAtmos precipitation fluxes (`pr`, `prra`, `prsn`) are sign-flipped so
precipitation shows positive
- The run-title caption wraps long tokens (e.g. commit hashes) and renders
newlines as line breaks

#### ClimaCoupler (multi-component) outputs
- `dashboard(path; coupled = true)` reads ClimaCoupler output directories
(`clima_atmos/`, `clima_land/`, `clima_ocean/`, `clima_seaice/` subfolders)
Expand Down
1 change: 1 addition & 0 deletions docs/src/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ precompute_dashboard_cache
default_obs
default_era5_obs
default_inversion_obs
default_gpcp_obs
```

## Index
Expand Down
9 changes: 7 additions & 2 deletions docs/src/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,19 @@ built-in registry ([`default_obs`](@ref)) covers:
|---|---|---|
| `lhf`, `shf`, `lwu`, `swu` | ERA5 monthly surface fluxes | ClimaLand names |
| `hfls`, `hfss`, `rlus`, `rsus` | ERA5 monthly surface fluxes | same fields, ClimaAtmos/CMIP names |
| `pr` | GPCP monthly precipitation | 2.5°×2.5°, 1979–present |
| `evspsbl` | ERA5 mean evaporation rate | sign flipped to positive-up |
| `prw` | ERA5 total column water | tcw includes ~1 % cloud condensate |
| `nee` | CarbonTracker CT2022 | inversion-derived, 2002–2020 |
| `gpp` | GOSIF-GPP v2 | |
| `er` | CarbonTracker + GOSIF residual | |
| `hr` | Hashimoto 2015 | |

Carbon fluxes are displayed in `g C m^-2 day^-1` (converted from the model's
`mol CO2 m^-2 s^-1`). All observation data ships as lazy artifacts — nothing
to download manually.
`mol CO2 m^-2 s^-1`), and water mass fluxes in `mm day^-1` (converted from
`kg m^-2 s^-1`, with the ClimaAtmos negative-downward precipitation fluxes
`pr`/`prra`/`prsn` sign-flipped so precipitation shows positive). All
observation data ships as lazy artifacts — nothing to download manually.

### Custom observations

Expand Down
2 changes: 1 addition & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ClimaViz is a Julia package for generating web dashboards from climate simulatio
## Features

- **Interactive web dashboards**: Generate web-based visualizations of your simulation outputs (see [The Dashboard](@ref))
- **Observation benchmarking**: Bias maps, metrics and a one-page model performance summary against ERA5, CarbonTracker, GOSIF and more (built-in artifacts)
- **Observation benchmarking**: Bias maps, metrics and a one-page model performance summary against ERA5, GPCP, CarbonTracker, GOSIF and more (built-in artifacts)
- **ClimaCoupler support**: Browse the atmos/land/ocean/seaice components of a coupled run from one dashboard (`coupled = true`)
- **HPC support**: Run directly from HPC environments with SSH port forwarding
- **Local development**: Test and visualize locally before deployment
Expand Down
2 changes: 1 addition & 1 deletion src/ClimaViz.jl
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ include("paramviz.jl")

export dashboard
export precompute_dashboard_cache
export default_obs, default_era5_obs, default_inversion_obs
export default_obs, default_era5_obs, default_inversion_obs, default_gpcp_obs
export Drivers, Parameters, Constants, Inputs, Output
export parameterisation, webapp, param_dashboard, dashboard_paramviz

Expand Down
177 changes: 147 additions & 30 deletions src/benchmark.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,26 @@ function _era5_monthly_nc_path()
end
end

function _era5_atmos_monthly_nc_path()
try
dir = artifact"era5_monthly_averages_atmos_single_level_1979_2024"
return joinpath(dir, "era5_monthly_averages_atmos_single_level_197901-202410.nc")
catch e
@warn "ClimaViz: ERA5 atmos monthly artifact not available; prw benchmark disabled" exception=(e, catch_backtrace())
return nothing
end
end

function _gpcp_monthly_nc_path()
try
dir = artifact"precipitation_obs"
return joinpath(dir, "precip.mon.mean.nc")
catch e
@warn "ClimaViz: GPCP precipitation artifact not available; pr benchmark disabled" exception=(e, catch_backtrace())
return nothing
end
end

function _era5_loader(varname_era5, varname_sim; flip_sign = false)
return function (_start_date)
path = _era5_monthly_nc_path()
Expand Down Expand Up @@ -52,6 +72,42 @@ fall back to the generic `"obs"`.
"""
obs_source_name(obs) = isnothing(obs) ? "obs" : get(obs.attributes, "obs_source", "obs")

# ERA5 `mer` (mean evaporation rate) uses the ECMWF sign convention — negative
# for evaporation — while ClimaAtmos `evspsbl` is positive for evaporation, so
# the conversion flips the sign while rescaling kg m⁻² s⁻¹ (≡ mm s⁻¹ of liquid
# water) to the dashboard's water-flux display unit (see `to_display_units`).
function _era5_evap_loader()
return function (_start_date)
path = _era5_monthly_nc_path()
isnothing(path) && return nothing
obs_var = ClimaAnalysis.OutputVar(path, "mer")
obs_var = ClimaAnalysis.convert_units(
obs_var,
_DISPLAY_WATER_UNITS;
conversion_function = u -> u * -_SECONDS_PER_DAY,
)
obs_var.attributes["short_name"] = "evspsbl"
obs_var.attributes["obs_source"] = "ERA5"
return obs_var
end
end

# ERA5 `tcw` is total column water (vapour + cloud condensate) whereas the sim
# `prw` is water-vapour path only; condensate is ~1 % of the column, so tcw is
# still a fair reference. The ERA5 monthly artifacts carry no tcwv field.
function _era5_prw_loader()
return function (_start_date)
path = _era5_atmos_monthly_nc_path()
isnothing(path) && return nothing
obs_var = ClimaAnalysis.OutputVar(path, "tcw")
(ClimaAnalysis.units(obs_var) == "kg m**-2") &&
(obs_var = ClimaAnalysis.set_units(obs_var, "kg m^-2"))
obs_var.attributes["short_name"] = "prw"
obs_var.attributes["obs_source"] = "ERA5"
return obs_var
end
end

"""
default_era5_obs()

Expand All @@ -62,19 +118,55 @@ monthly observation loaders. Each loader takes a `start_date` and returns a
Covers the surface fluxes (W m⁻², monthly, 1°×1°) under both the ClimaLand
names (`lhf`, `shf`, `lwu`, `swu`) and the ClimaAtmos/CMIP names (`hfls`,
`hfss`, `rlus`, `rsus`) — the same ERA5 fields and sign conventions either
way. Returns an empty Dict if the underlying artifact is unavailable.
way — plus evaporation (`evspsbl`, from `mer`, in the mm day⁻¹ display unit)
and water-vapour path (`prw`, from `tcw`, kg m⁻²). Returns an empty Dict if
the underlying artifacts are unavailable.
"""
function default_era5_obs()
isnothing(_era5_monthly_nc_path()) && return Dict{String, Function}()
obs = Dict{String, Function}()
if !isnothing(_era5_monthly_nc_path())
merge!(
obs,
Dict{String, Function}(
"lhf" => _era5_loader("mslhf", "lhf"; flip_sign = true),
"shf" => _era5_loader("msshf", "shf"; flip_sign = true),
"lwu" => _era5_loader("msuwlwrf", "lwu"; flip_sign = false),
"swu" => _era5_loader("msuwswrf", "swu"; flip_sign = false),
"hfls" => _era5_loader("mslhf", "hfls"; flip_sign = true),
"hfss" => _era5_loader("msshf", "hfss"; flip_sign = true),
"rlus" => _era5_loader("msuwlwrf", "rlus"; flip_sign = false),
"rsus" => _era5_loader("msuwswrf", "rsus"; flip_sign = false),
"evspsbl" => _era5_evap_loader(),
),
)
end
isnothing(_era5_atmos_monthly_nc_path()) ||
(obs["prw"] = _era5_prw_loader())
return obs
end

"""
default_gpcp_obs()

Built-in `Dict{String, Function}` mapping `pr` to a GPCP monthly precipitation
loader (2.5°×2.5°, 1979–present, from the `precipitation_obs` artifact). GPCP
stores mm/day — the dashboard's water-flux display unit, into which the sim
`pr` is converted by [`to_display_units`](@ref) — so only the unit string is
normalized. Returns an empty Dict if the underlying artifact is unavailable.
"""
function default_gpcp_obs()
isnothing(_gpcp_monthly_nc_path()) && return Dict{String, Function}()
return Dict{String, Function}(
"lhf" => _era5_loader("mslhf", "lhf"; flip_sign = true),
"shf" => _era5_loader("msshf", "shf"; flip_sign = true),
"lwu" => _era5_loader("msuwlwrf", "lwu"; flip_sign = false),
"swu" => _era5_loader("msuwswrf", "swu"; flip_sign = false),
"hfls" => _era5_loader("mslhf", "hfls"; flip_sign = true),
"hfss" => _era5_loader("msshf", "hfss"; flip_sign = true),
"rlus" => _era5_loader("msuwlwrf", "rlus"; flip_sign = false),
"rsus" => _era5_loader("msuwswrf", "rsus"; flip_sign = false),
"pr" => function (_start_date)
path = _gpcp_monthly_nc_path()
isnothing(path) && return nothing
obs_var = ClimaAnalysis.OutputVar(path, "precip")
obs_var = ClimaAnalysis.replace(obs_var, missing => NaN)
obs_var = ClimaAnalysis.set_units(obs_var, _DISPLAY_WATER_UNITS)
obs_var.attributes["short_name"] = "pr"
obs_var.attributes["obs_source"] = "GPCP"
return obs_var
end,
)
end

Expand All @@ -96,32 +188,54 @@ const _SECONDS_PER_DAY = 86400.0

# ─── Display-unit conversion ──────────────────────────────────────────────────
#
# ClimaLand stores carbon fluxes in mol CO2 m^-2 s^-1, where values are ~1e-6 and
# the metrics table would render every cell as 0.00. For display we rescale them —
# and only them — to the more legible g C m^-2 day^-1. Detection is by unit string
# so every carbon flux (gpp/nee/er/hr/ra, with or without an observation) is
# caught while energy (W m^-2) and water (kg m^-2 s^-1) fluxes pass through.
# ClimaLand stores carbon fluxes in mol CO2 m^-2 s^-1 and the atmos/land models
# store water fluxes in kg m^-2 s^-1, where values are ~1e-6 and the metrics
# table would render every cell as 0.00. For display we rescale them — and only
# them — to the more legible g C m^-2 day^-1 and mm day^-1. Detection is by unit
# string so every such flux (with or without an observation) is caught while
# energy fluxes (W m^-2) pass through.

const _CARBON_FLUX_UNITS = ("mol CO2 m^-2 s^-1", "mol CO2 m**-2 s**-1")
const _DISPLAY_CARBON_UNITS = "g C m^-2 day^-1"

is_carbon_flux(var) = ClimaAnalysis.units(var) in _CARBON_FLUX_UNITS

const _WATER_FLUX_UNITS = ("kg m^-2 s^-1", "kg m**-2 s**-1")
const _DISPLAY_WATER_UNITS = "mm day^-1"

# ClimaAtmos reports precipitation-type fluxes as negative downward mass
# fluxes; flip them for display so precipitation is positive, matching GPCP
# and everyday convention (same flip as ClimaCoupler's leaderboard).
const _NEGATIVE_DOWN_WATER_FLUXES = ("pr", "prra", "prsn")

is_water_flux(var) = ClimaAnalysis.units(var) in _WATER_FLUX_UNITS

"""
to_display_units(var)

Convert a freshly-loaded simulation `OutputVar` to the units the dashboard
displays. Currently this rescales carbon fluxes from `mol CO2 m^-2 s^-1` to
`g C m^-2 day^-1` (× molar mass of C × seconds/day); all other variables are
returned unchanged. Apply this once, at each point a sim variable is loaded, so
the map, bias panel, time series, and metrics all share one consistent unit.
displays. Carbon fluxes are rescaled from `mol CO2 m^-2 s^-1` to
`g C m^-2 day^-1` (× molar mass of C × seconds/day); water fluxes from
`kg m^-2 s^-1` (≡ mm s⁻¹ of liquid water) to `mm day^-1` (× seconds/day, with
a sign flip for the negative-downward precipitation fluxes `pr`/`prra`/`prsn`);
all other variables are returned unchanged. Apply this once, at each point a
sim variable is loaded, so the map, bias panel, time series, and metrics all
share one consistent unit.
"""
function to_display_units(var)
is_carbon_flux(var) || return var
return ClimaAnalysis.convert_units(
var, _DISPLAY_CARBON_UNITS;
conversion_function = u -> u * _G_C_PER_MOL * _SECONDS_PER_DAY,
)
if is_carbon_flux(var)
return ClimaAnalysis.convert_units(
var, _DISPLAY_CARBON_UNITS;
conversion_function = u -> u * _G_C_PER_MOL * _SECONDS_PER_DAY,
)
elseif is_water_flux(var)
sign = get(var.attributes, "short_name", "") in _NEGATIVE_DOWN_WATER_FLUXES ? -1.0 : 1.0
return ClimaAnalysis.convert_units(
var, _DISPLAY_WATER_UNITS;
conversion_function = u -> u * sign * _SECONDS_PER_DAY,
)
end
return var
end

function _inversion_nc_path()
Expand Down Expand Up @@ -192,13 +306,16 @@ end
"""
default_obs()

Combined built-in observation set: ERA5 monthly energy fluxes
([`default_era5_obs`](@ref): `lhf`, `shf`, `lwu`, `swu`) merged with the
inversion-derived carbon fluxes ([`default_inversion_obs`](@ref): `nee`, `gpp`,
`er`, `hr`). This is the default observation registry for [`dashboard`](@ref)
and [`precompute_dashboard_cache`](@ref).
Combined built-in observation set: ERA5 monthly energy/water fluxes and
water-vapour path ([`default_era5_obs`](@ref): `lhf`, `shf`, `lwu`, `swu` and
their CMIP aliases, `evspsbl`, `prw`), the inversion-derived carbon fluxes
([`default_inversion_obs`](@ref): `nee`, `gpp`, `er`, `hr`), and GPCP
precipitation ([`default_gpcp_obs`](@ref): `pr`). This is the default
observation registry for [`dashboard`](@ref) and
[`precompute_dashboard_cache`](@ref).
"""
default_obs() = merge(default_era5_obs(), default_inversion_obs())
default_obs() =
merge(default_era5_obs(), default_inversion_obs(), default_gpcp_obs())

mutable struct ObsBundle
loaders::Dict{String, Function}
Expand Down
6 changes: 5 additions & 1 deletion src/layout.jl
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,12 @@ function layout(

# Optional run caption shown just below the play button.
run_title_style = Bonito.Styles(
"font-size" => "1.0rem", "font-weight" => "700",
"font-size" => "0.85rem", "font-weight" => "700",
"margin" => "8px 0 0 0", "text-align" => "center", "color" => "#9ec5fe",
# Render newlines in the caption as line breaks (multi-line run titles),
# and let an over-long token (e.g. a 40-char commit hash) wrap inside the
# 400px menu column instead of overflowing it.
"white-space" => "pre-line", "overflow-wrap" => "anywhere",
)

menu_items = Any[dark_mode_row]
Expand Down
1 change: 1 addition & 0 deletions test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import ClimaAnalysis

@testset "ClimaViz.jl" begin
include("test_utils.jl")
include("test_benchmark.jl")
include("test_cache.jl")
include("test_coupler.jl")
end
Loading
Loading