From b75312bfa6482b3419e230120e25f0ad6b64f6f8 Mon Sep 17 00:00:00 2001 From: Tapio Schneider Date: Mon, 13 Jul 2026 22:59:05 -0700 Subject: [PATCH 1/2] Lock the copy-free flux-getter contract (docs + regression test) The output flux getters (`net_flux`, `lw_flux_up`, `sw_flux_dn`, ...) return single-level views of plain `(nlev, ncol)` presentation buffers, so a host wraps them with ClimaCore's `array2field` copy-free and in the correct memory order. A consumer that `copy(getter(solver))`s before wrapping materializes a full `(nlev, ncol)` field per getter per call; a diagnostics pass reading many fluxes every step turns that into large per-step allocations (observed as ~17 MB/step in a ClimaAtmos aquaplanet flame test, from ~12 such copies). RRTMGP itself is unchanged and its solve stays allocation-free; this documents the getter contract and adds a regression test so the copy-free guarantee can't silently regress. Co-Authored-By: Claude Opus 4.8 --- NEWS.md | 8 +++++++ docs/src/getters.md | 13 ++++++++++-- test/standalone.jl | 52 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/NEWS.md b/NEWS.md index a1bc44dba..da9f19053 100644 --- a/NEWS.md +++ b/NEWS.md @@ -3,6 +3,14 @@ RRTMGP.jl Release Notes main ------ +- Documented and added a regression test for the **copy-free flux-getter + contract**: the output flux getters (`net_flux`, `lw_flux_up`, `sw_flux_dn`, + …) return single-level views of plain `(nlev, ncol)` presentation buffers, so + a host wraps them with ClimaCore's `array2field` without copying and in the + correct memory order. Hosts must not `copy(getter(solver))` before wrapping — + that materializes a full field per getter per call, which for a diagnostics + pass reading many fluxes each step caused large per-step allocations. No + behavior change; getters and buffers are unchanged. v0.22.0 ------- diff --git a/docs/src/getters.md b/docs/src/getters.md index 3c29af390..527e0e728 100644 --- a/docs/src/getters.md +++ b/docs/src/getters.md @@ -51,8 +51,17 @@ F = RRTMGP.net_flux(solver) # (nlev, ncol) view of the resu ``` With ClimaCore, this forms the bridge: `array2field(getter(solver), space)` -wraps an input view as a `Field` (writes flow into the solver's buffer), and -`field2array(dst) .= getter(solver)` copies an output into a destination field. +wraps a getter's view as a `Field` — for an input, writes flow into the solver's +buffer; for an output, the `Field` reads the buffer in place — and +`field2array(dst) .= getter(solver)` copies an output into an existing +destination field. Both directions are copy-free: the getters are single-level +views of plain `(nlev, ncol)` buffers, so `array2field`'s lazy `reshape` wraps +them without materializing anything. Do **not** `copy(getter(solver))` before +wrapping — the view is directly wrappable, and copying allocates a full +`(nlev, ncol)` field on every call (reading many fluxes per step this way adds +up to large per-step allocations). Likewise avoid `parent(getter(solver))` as a +shortcut: `parent` returns the full boundary-extended buffer, undoing the domain +masking. On the GPU, the returned views are plain `SubArray`s of device arrays; to bring one to the CPU, use `Array(getter(solver))` (or the element-wise `.=` copy above). One subtlety: the solver's compute buffers are indexed diff --git a/test/standalone.jl b/test/standalone.jl index 87dc6b2cb..20036fd67 100644 --- a/test/standalone.jl +++ b/test/standalone.jl @@ -354,3 +354,55 @@ end @test (@allocated RRTMGP.update_net_fluxes!(solver)) == 0 end end + +# The output flux getters return plain-backed `(nlev, ncol)` views (single-level +# `SubArray`s of the presentation buffers), so a host can wrap them with ClimaCore's +# `array2field` (a lazy `reshape`) copy-free and in the correct memory order. Regression +# guard: a consumer must not `copy(getter(solver))` before wrapping — that materializes a +# full `(nlev, ncol)` field per getter per call, which for a diagnostics pass reading many +# fluxes every step adds up to large per-step allocations. Assert the views are plain-backed +# and that reading one into a host buffer is allocation-free. +@testset "flux getters are copy-free readable (host contract)" begin + context = ClimaComms.context() + device = ClimaComms.device(context) + DA = ClimaComms.array_type(device) + solver = RRTMGP.solve_gray(Float64; nlay = 60, ncol = 10).solver + RRTMGP.update_fluxes!(solver) # populate the presentation buffers the getters read + flux_getters = ( + RRTMGP.net_flux, + RRTMGP.lw_flux_up, + RRTMGP.lw_flux_dn, + RRTMGP.lw_flux_net, + RRTMGP.sw_flux_up, + RRTMGP.sw_flux_dn, + RRTMGP.sw_flux_net, + RRTMGP.sw_direct_flux_dn, + ) + for getter in flux_getters + g = getter(solver) + # A single-level view of a plain buffer: `parent` is the buffer itself (not a + # nested `SubArray` or a `PermutedDimsArray`), so `reshape`/`array2field` wraps it + # without a materializing copy and without reordering memory. + @test g isa SubArray + @test parent(g) isa DA + end + # (The all-sky and clear-sky flux getters share the identical presentation-buffer + # backing; the all-sky path is exercised in test/all_sky_with_aerosols_utils.jl.) + # + # Reading a getter into a host buffer must not materialize a full field, whereas + # `copy(getter)` does — that contrast is the whole point. Assert it directly (a + # version-robust relative bound; the in-place broadcast carries only small, constant + # SubArray overhead, far below a full `(nlev, ncol)` copy). + if device isa ClimaComms.CPUSingleThreaded + for getter in flux_getters + g = getter(solver) + dst = Array(g) # host buffer of matching shape + copy(g) # warm up / compile + dst .= g + full = @allocated copy(g) # copy materializes a full field + read = @allocated (dst .= g) # reading needs no such copy + @test full >= length(g) * sizeof(eltype(g)) + @test read < full ÷ 4 + end + end +end From 3108bd0acb6372c0822f33bd528657c6b5eafc6c Mon Sep 17 00:00:00 2001 From: Tapio Schneider Date: Tue, 14 Jul 2026 07:25:39 -0700 Subject: [PATCH 2/2] Release v0.22.1 Co-Authored-By: Claude Fable 5 --- NEWS.md | 3 +++ Project.toml | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/NEWS.md b/NEWS.md index da9f19053..f30e5d881 100644 --- a/NEWS.md +++ b/NEWS.md @@ -3,6 +3,9 @@ RRTMGP.jl Release Notes main ------ + +v0.22.1 +------- - Documented and added a regression test for the **copy-free flux-getter contract**: the output flux getters (`net_flux`, `lw_flux_up`, `sw_flux_dn`, …) return single-level views of plain `(nlev, ncol)` presentation buffers, so diff --git a/Project.toml b/Project.toml index 042ef4fcd..82680c50b 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "RRTMGP" uuid = "a01a1ee8-cea4-48fc-987c-fc7878d79da1" authors = ["Climate Modeling Alliance"] -version = "0.22.0" +version = "0.22.1" [deps] Adapt = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"