Skip to content

Fast-JX: correct bin 17-18 tables (TOA flux, Rayleigh, O3 attenuation) to Cloud-J v8.0 — fixes J_NO2 ~2x low - #234

Open
xk-y wants to merge 5 commits into
EarthSciML:mainfrom
xk-y:fastjx-v8-flux-bins-17-18
Open

Fast-JX: correct bin 17-18 tables (TOA flux, Rayleigh, O3 attenuation) to Cloud-J v8.0 — fixes J_NO2 ~2x low#234
xk-y wants to merge 5 commits into
EarthSciML:mainfrom
xk-y:fastjx-v8-flux-bins-17-18

Conversation

@xk-y

@xk-y xk-y commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fast-JX's 18-bin photolysis cross-sections are Cloud-J v8.0 (the NO2 300 K row and the NO3 190/298c rows match FJX_spec.dat digit-for-digit), but the TOA solar flux top_flux bins 17–18 still held legacy Fast-JX v7.x values (1.547e16 / 2.131e17). Cloud-J v8.0 restructured bins 17–18 to 345-485-778 nm with fluxes 4.721e16 / 1.488e17.

Because bin 17 (429 nm) contributes ~80% of the J_NO2 integral, the stale flux made J_NO2 ≈ 0.47× the physical value, biasing daytime O₃ low. It also affected J_NO3 (whose v8.0 σ lives entirely in bins 17–18).

Change

Two numeric literals in the existing const top_flux = SA_F32[…] array (bins 17–18 → v8.0). Nothing else.

Why it's safe / minimal

  • A scan of all 18-bin cross-sections shows only NO2, NO3, and stratospheric halogens (OClO/BrO/HOBr) have non-negligible bin 17–18 absorption — O₃/HCHO/H2O2/etc. are ≈0 there, so no UV J-value changes.
  • The WL effective-wavelength labels are left at 380/574 on purpose: WL is used only to name the F_<wl> actinic-flux variables (never in any J-value computation), and those names appear in golden test references — renaming would break tests for zero physics benefit.

Verification

From-scratch integration (φ=1, TOA): J_NO2 4.26e-3 → 9.17e-3 s⁻¹ (physical clear-sky TOA ≈ 9–10e-3); J_NO3 lands at a physical 0.24 s⁻¹.

Found by a module-by-module equivalence audit of this stack vs GEOS-Chem Classic 14.7.1 / Cloud-J v8.0. Part of the Stage 6 addendum in #225.


Update (2nd commit): the same v7.x/v8.0 generation mismatch existed in the bin 17–18 attenuation tables — σ_Raylay[17:18] (Rayleigh ∝ λ⁻⁴; legacy 380/574 nm values over-attenuate the corrected bin-17 flux by ~14% at the surface) and the σ_O3 attenuation rows (Chappuis band at 429/631 nm). Both are now also set to the v8.0 values, completing the bin-17/18 fix. Pinned test values recomputed with the full package; bins 1–16 verified unchanged, and the bin-17/18 changes reproduce the analytic expectation (flux ratio × exp(−Δτ·AMF) = 4.34 / 0.7146) to 3 decimals.

Note for downstream users: FastJX_interpolation_troposphere reads pre-baked fluxes from tropospheric_interpolation_data.bson, which caches values computed with the old tables — that BSON must be regenerated for the interpolation path to pick up this fix (the online FastJX() path picks it up immediately).

Update (3rd commit): the BSON is now regenerated in this PR, so the note above is discharged.
Leaving it out would have made this PR inert for every configuration on the interpolated path —
FastJX_interpolation_troposphere never consults top_flux.

The table ships as a binary with no committed generator, but it reproduces this package's own
direct-beam kernel digit-for-digit at the surface row, so bins 17–18 are moved across by pointwise
ratio transfer rather than by re-deriving the (lost) aloft methodology:

Z_new = Z_old · K_new / K_old        K = direct_solar_beam_box, legacy vs v8.0 bin tables

Checks: bands 1–16 bit-identical; the two kernels' geometric zero sets coincide exactly; wherever
K_old ≠ 0 the transfer is exact to 3.1e-16; key/type layout unchanged so existing BSON.@load
calls are unaffected. Result on the interpolated path at the surface, cosSZA = 1, 298 K:
J_NO2 2.19e-3 → 6.06e-3 s⁻¹.

One residual is deliberately left in place: at a single node per band (P = 50500 Pa,
cosSZA = −0.04, sun ~2.3° below the horizon) both kernels evaluate to zero while the table does
not, so the ratio is undefined there and the legacy entry stays. σ_NO2[18] = 0, so J_NO2 is
untouched; only bin-18 species see it, only in twilight cells.

Worth adding, but not in this PR: a daytime J assertion. Nothing in the suite could have caught
the omission above. fastjx_test.jl only asserts mtkcompile succeeds on the interpolated
operator; geoschem_test.jl checks equation strings; and both composition goldens
(compose_fastjx_superfast_test.jl, pollu_test.jl) sample index 4320 of a 24 h solve, which at
the operators' default coordinates (lat 40 N, long −97 W, t_ref = 0) is cosSZA = −0.35 — the
middle of the night, where every J is identically zero. Changing the table by 2.77× moves
sol_middle by 1.7e-12. O₃ is not a usable probe there either: over the whole 24 h it ranges only
39.9996 … 40.0, so day and night differ by 4e-6 relative, well inside the 1e-4 tolerance. A useful
assertion reads the J values directly at an index that is actually in daylight — j_NO2 (bin-17
dominated) and j_H2O2 (UV bins, independent of this change), plus the night zeros. Happy to send
it as a separate test-only PR; keeping it out of here so this one stays single-topic.

…2 ~2x low)

The 18-bin photolysis cross-sections in Fast-JX.jl are Cloud-J v8.0 (the NO2 300K
and NO3 190/298c rows match FJX_spec.dat digit-for-digit), but top_flux bins 17-18
still held legacy Fast-JX v7.x values (1.547e16 / 2.131e17). Cloud-J v8.0
restructured bins 17-18 to 345-485-778 nm with fluxes 4.721e16 / 1.488e17.

Bin 17 (429 nm) contributes ~80% of the J_NO2 integral, so the stale flux made
J_NO2 ~0.47x the physical value (TOA, phi=1: 4.26e-3 -> 9.17e-3 s^-1); it also
affects J_NO3. Only NO2/NO3 and stratospheric halogens have non-negligible
bin 17-18 cross-sections, so no UV J-values change. WL labels are left at 380/574
(used only to name the F_<wl> variables, not in any J-value computation) to avoid
renaming variables referenced by golden tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mpletes the bin-17/18 generation fix)

Same generation mismatch as the top_flux fix in this PR, in the attenuation
tables: sigma_Raylay[17:18] and the sigma_O3 attenuation rows held legacy
values for the v7.x 380/574 nm bins, while the flux (after this PR) and all
photolysis cross-sections are Cloud-J v8.0 (429/631 nm bins):

  sigma_Raylay bin17: 2.087e-26 -> 1.387e-26  (Rayleigh ~ lambda^-4;
                      (429/380)^-4 = 0.616 ~ observed 0.665, flux-weighted)
  sigma_Raylay bin18: 3.848e-27 -> 3.136e-27
  sigma_O3    bin17: 6.167e-23 -> 2.125e-22  (Chappuis band at 429 vs 380 nm;
                      v8 value is T-independent, applied to all three T rows)
  sigma_O3    bin18: 1.666e-21 -> 2.325e-21

Without this, the legacy 380-nm Rayleigh over-attenuates the corrected bin-17
flux by ~14% at the surface (SZA=30), leaving J_NO2 low even after the flux
fix. Pinned test values recomputed with the full package (bins 1-16 unchanged,
verifying no other bin is touched); the bin-17 change reproduces the analytic
expectation flux_ratio x exp(-dtau*AMF) = 4.34 and bin-18 = 0.7146 to 3
decimal places.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@xk-y xk-y changed the title Fast-JX: correct bin 17-18 TOA solar flux to Cloud-J v8.0 (fixes J_NO2 ~2x low) Fast-JX: correct bin 17-18 tables (TOA flux, Rayleigh, O3 attenuation) to Cloud-J v8.0 — fixes J_NO2 ~2x low Jul 17, 2026
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.42%. Comparing base (4cb6241) to head (4b94ca0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #234   +/-   ##
=======================================
  Coverage   87.42%   87.42%           
=======================================
  Files          20       20           
  Lines        1463     1463           
=======================================
  Hits         1279     1279           
  Misses        184      184           
Flag Coverage Δ
docs 71.26% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

xk-y and others added 3 commits July 17, 2026 12:18
…e CI)

The bin-17/18 flux+Rayleigh+O3 correction changes every J golden with nonzero
bin-17/18 sigma*phi, but the previous commit updated only the two Direct Flux
testitems it directly edited. Regenerated the remaining stale goldens by
running the branch's own functions with each test's exact inputs:
  u_4 (NO2)     4.261e-3 -> 9.168e-3  (the intended ~2.15x J_NO2 fix)
  u_2 (H2COb)   +3.0%    | u_3 (CH3OOH) +1.3% | u_1 (H2O2) +0.2%
  Twilight bin-18 pins: P=1 2.097e17 -> 1.461e17; P=100 4.909e16 -> 3.275e16
u_0 (O31D) and u_2a (H2COa) unchanged (their bin-17 sigma is 0). Caught by an
adversarial self-review of this PR; CI Tests were red without this.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Production (FastJX_interpolation_troposphere) reads actinic fluxes from
tropospheric_interpolation_data.bson, not from top_flux/direct_flux.jl, so the
bin-17/18 correction must also land in the table or it has no effect on the CTM.

The table has no committed generator (added upstream in a1c16bc without a
script). It matches the deployed direct-beam kernel digit-for-digit at the
surface row but embeds a lost methodology aloft, so instead of re-running an
unavailable generator, bins 17-18 are updated by pointwise ratio transfer on
the 23P x 61cosSZA grid:

    Z_new = Z_old * K_new(P,cosSZA) / K_old(P,cosSZA)

where K_* is this package's own direct-beam kernel (direct_solar_beam_box)
evaluated with the legacy vs Cloud-J v8.0 bin-17/18 tables (top_flux,
sigma_Rayleigh, sigma_O3). The v8.0 flux swap is a pure multiplier of the
Beer-Lambert attenuation, so it transfers exactly; the small sigma-driven
attenuation change modulates it; the table's aloft behaviour is preserved.

Checks (scripted): bands 1-16 bit-identical; geometric zero sets of old/new
kernels coincide on the grid (no created/destroyed zeros); surface row equals
the new kernel to machine epsilon; key/type layout unchanged.

Effect on the table path at the surface (cosSZA=1, 298 K): J_NO2
2.19e-3 -> 6.06e-3 s^-1 (2.77x; TOA flux-mismatch factor 2.15x plus weaker
429-nm-vs-380-nm Rayleigh attenuation). The remaining gap to the Cloud-J
reference (~8.6e-3) is the missing scattered/diffuse component, addressed
separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adversarial review of the branch flagged two places where the comments claim
more than was verified:

1. The top_flux header read "Cloud-J v8.0 SPhot" for the whole 18-bin block,
   but only bins 17-18 were changed and checked against v8.0's FJX_spec.dat.
   Bins 1-16 are the pre-existing table and differ from the v8.0 SPhot rows by
   up to ~0.5% (bin 6: 4.68e12 vs 4.655e12; bin 4: 9.278e11 vs 9.238e11), so a
   later maintainer diffing only 17-18 would wrongly mark the table validated.

2. The WL note called the 380/574 labels a non-issue because WL is naming-only.
   Naming-only is right, but the names are now wrong: bins 17-18 carry the v8.0
   345-485 / 485-778 nm bands (effective 429 / 631 nm), so the exported F_380 /
   F_574 variables mislabel their own bands by +49 / +57 nm. Renaming them to
   F_429 / F_631 is the correct fix and is left as a follow-up because it is a
   breaking interface change and does not belong in a numerical bug fix.

Comments only; no numeric or structural change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant