Batch C — Science Demos II: 8 differentiable-inference demos + harness by drannarosen · Pull Request #7 · jaxstro/progenax

drannarosen · 2026-06-13T04:48:34Z

Summary

Eight gated-CLI science demos across the four undemoed threads (binaries, environment IMF, structural recovery, differentiable diagnostics), scoped as paper seeds + methods-table entries — each a gated CLI (exit 0 = all gates pass) + a website page + a measured docstring run-record, built on (and extending) the Batch-B physics-direct inference harness.

Demo	Type	Headline result
B9 binary energy budget	methods+	Q_com=0.5000 clean; \|E_internal\| dwarfs \|W_com\| 12–1900×; EFF-vs-King controlled comparison (identical E_internal at the same key)
B11 King concentration	methods	recover (W₀, r_c) from binned star counts alone; honest ρ(W₀,r_c)=−0.91 degeneracy — validates the new Poisson channel
B10 diff diagnostics	methods	q_approx tracks exact Q to <0.06 (substructure regime); Λ_approx rank-faithful; AD-vs-FD 4%
B5 birth environment	paper	α₃ measurable (~40σ, CRLB empirically validated); environment rank-1 unrecoverable
B4 binary mass function	paper	ignoring the Moe P–q coupling biases f_b −3.6σ; Tout+1996 ZAMS photometry via fluxax
B6 anisotropy	paper	OM r_a recovered; OM fit to a Michie cluster → 12.9× χ² inflation + r_a mis-estimate (formalism matters)
B7 tidal radius (stretch)	paper	r_t from the count-limited outskirts (93% of the Fisher info from the outer bins) → Galactocentric distance R_gal = 2.79±0.01 kpc
B8 rotation + projection (stretch)	paper	⟨v_los⟩ measures the product ω·sin(i), not ω: the (ω,i) Fisher is rank-1 — the realism-axis bridge

Harness

Poisson number-density channel in scripts/_demo_inference.py — binned_number_density / poisson_loglike / poisson_fisher_information (reverse-mode J^T diag(1/μ) J, ODE-custom_vjp-safe), the Poisson sibling of the σ-channel trio. Used by B11 and B7.
poisson_fisher_information floors μ (like poisson_loglike) so a truncation fit's empty bins are NaN-safe (B7).
30/30 _demo_inference unit tests (tests/experimental/unit/).

Infrastructure / fixes

fluxax wired CI-safely (B4 Tout ZAMS): the private sibling is not locked — documented [experimental] note + guarded import + local uv pip install -e ../fluxax --no-deps. uv.lock unchanged; uv lock --check green (mirrors the audit's jaxstroviz handling — does not re-break the experimental CI job).
docs/website/Makefile (make build/make start) sets NODE_OPTIONS=--no-deprecation to silence the upstream mystmd url.parse() warning.
Latent Batch-B gap fixed: the *.png gitignore exception covered 50-validation/figures/ but not 60-science-demos/figures/, so the B1–B3 demo figures were never committed — fixed + orphaned figures tracked.
peuten-2017 YAML frontmatter quoted; Tout1996 and King1962 bib entries added.

Two physics/harness catches (with regression tests)

A hard jnp.where(r < r_t, ρ, 0) truncation has zero gradient through its boundary condition → r_t was non-differentiable (B7). Fixed by integrating the untruncated cumulative once and clipping bin edges at r_t.
The (a, r_t) and (ω, i) inverse problems are genuinely rank-deficient — surfaced honestly rather than masked.

Verification

All 8 demos exit 0 (run-records captured); every figure visually inspected (publication-style, physically correct).
myst: 159 pages, 0 warnings (make build).
Released-core gate (FAST, -m "not slow", -n auto): 1221 passed, 1 skipped, exit 0 (1192 released-core + 29 harness). Batch C touched only scripts/ + docs/ + a pyproject comment — src/ untouched all batch.

🤖 Generated with Claude Code

… C harness) Adds the number-density observable channel as the Poisson sibling of the existing sigma-channel trio, for the B11 King-concentration and B7 tidal-radius demos: - binned_number_density(pos, r_edges): frozen per-shell counts (the data). - poisson_loglike((counts, weight), predict_mu_fn): per-bin Poisson log-pmf sum w_k (N_k log mu_k - mu_k); gradient through the model only; mu floored for the log (clean subgradient, NaN-free; populated-bin mu->0 still penalized). - poisson_fisher_information(predict_mu_fn, z_hat, weight): reverse-mode (jacrev) expected Fisher J^T diag(w/mu) J -- ODE-safe through the King diffrax custom_vjp that crashes jax.hessian (same reason as fisher_information_gn). Poisson (not Gaussian-on-log-count) so low-occupancy outer bins carry honest counting errors -- the regime that constrains a truncation radius. 11 new unit tests (counts, analytic-MLE gradient, Adam recovery, zero-count NaN-safety, weight masking, closed-form Fisher); 28/28 in the file pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The *.png gitignore exception covered 50-validation/figures/ but NOT 60-science-demos/figures/, so the B1-B3 demo figures (cross_engine, halo_core, imf_equip_*) referenced by the rendered pages were never committed -- a clean checkout / deploy build would have broken every science-demo image. Mirror the 50-validation exception for 60-science-demos and commit the orphaned figures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ir (Batch C) Methods + mini-science demo for binaries.diagnostics.binary_energy_budget. build_binary_cluster virializes the system COMs to Q=0.5 treating binaries as point masses (McLuster scale separation); the internal binary binding is a separate reservoir Q never touches. The demo makes that explicit: - EFF young-cluster primary (the physically apt birth profile for *primordial* binaries), Maschberger primary IMF (smooth/differentiable). Two controlled sweeps: hardness (LogUniformPeriod band centre) and binary fraction f_b. - Robust gated facts: Q_com = 0.5000 at every point; |E_internal| dwarfs |W_com| (ratio 11.6-1919x) and grows 3 orders as binaries harden. - Q_resolved is reported as a CONTAMINATED diagnostic (scatters around 0.5 with orbital phase), NOT gated as a monotonic deflation -- the honest physics: a hard binary's internal virial is itself ~0.5. - Environment figure (its own): the SAME Moe & Di Stefano population in EFF vs King(W0=7) at the SAME key -> E_internal byte-identical (2.965e5); the more bound King has larger |W_com| -> reservoir fraction 440 (EFF) vs 298 (King). ALL PASS (2026-06-12, N_sys=2000, ~32 s). Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The unquoted description contained 'II: Multimass' (colon-space parses as a YAML mapping) and embedded double-quotes, so myst reported invalid frontmatter. Single-quote the scalar. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The methods showcase for the new Poisson number-density channel: recover a single-population King model's (W0, r_c) from binned radial STAR COUNTS alone (no kinematics) -- the observable a photometric survey delivers. - Forward model: Engine A single King-limit LIMEPY component (g=1), differentiable in (W0, r_c) through psi_grid; King density is the closed form limepy_density_hat(W(r), g=1); beyond r_t the density is exactly 0 (the truncation that pins the outer profile). - mu_k = N_obs * (enclosed fraction in bin k); per-bin Poisson loglike; Fisher via the reverse-mode poisson_fisher_information (ODE custom_vjp crashes jax.hessian). - ALL PASS (2026-06-12, N=30000, ~29 s): W0 6.0->6.028+-0.024 (+1.15s), r_c 1.0->0.995+-0.010 (-0.47s); 3 inits -> identical loss; c=log10(r_t/r_c) MLE 1.263 vs truth 1.255 (King 1966 Tab. II). - Honest finding foregrounded: Fisher rho(W0,r_c) = -0.91 -- the count channel alone is strongly W0-r_c DEGENERATE; both marginals tight, but it's exactly the degeneracy the velocity channel (B2/B3) breaks. Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Validate progenax's two differentiable structure surrogates against their exact combinatorial oracles, and show the substructure surrogate is a faithful loss: - q_approx vs exact compute_q_parameter (CW04 Q): tracks to |diff|<0.06 in the substructure regime (Q<0.8) it is used for, preserves ordering clumpy<uniform< concentrated; degrades mildly at high concentration; Plummer (Q>1) excluded as outside the CW04-calibrated range (honest scope). - lambda_msr_approx vs exact compute_lambda_msr (Allison Lambda_MSR): rank-faithful (Spearman 1.00) across a segregation sweep, magnitude-compressed in the soft regime, converges to exact as tau,beta->0; unsegregated null ~1. - differentiability: q(p)=q_approx(u^p D), autodiff dq/dp matches finite-difference to 4% at h=1e-3 -- usable as a gradient objective. Honest scope stated: smooth/radial + simple-clump axis only; full fractal- substructure inference lives in the experimental gravoturb_fdf package. ALL PASS (2026-06-12, N=400, ~6 s). Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

`make build` / `make start` set NODE_OPTIONS=--no-deprecation to silence the upstream mystmd url.parse() DEP0169 warning (mystmd 1.10.1 is the latest; the deprecated call is in mystmd/its deps, not fixable here). Targeted so genuine node warnings still surface. Verified: 0 deprecation lines, 154 pages built. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Inference demo on the environment-dependent IMF (BirthEnvironment + env_to_imf_params, Marks+2012 / Jerabkova+2018). The honest two-part result: - WHAT YOU CAN read off the masses: alpha3. A top-heavy metal-poor massive truth ([Fe/H]=-1.5, logM_ecl=6.5, sfe=0.3 -> alpha3=1.625) is recovered cleanly (1.6249+-0.0054, +0.04s). sigma(alpha3) ~ N^-1/2, with the CRLB validated EMPIRICALLY (resampled scatter 0.0144 vs analytic 0.0170 @ N=1e4); ~40-sigma top-heavy detection at N=1e4. - WHAT YOU CANNOT: the birth environment. The (FeH, logM, sfe) -> alpha3 map is 3-to-1, so the environment-space Fisher is RANK 1 (eigenvalues [-1.8e-12, 1.8e-12, 1.4e4], cond 1.4e304) -- two flat directions. A continuum of environments gives the same alpha3; recovering any one birth parameter needs an external prior on the other two. Mass channel only (no ODE/kinematics); fisher_cov (ODE-free). ALL PASS (2026-06-12, ~7 s). Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… (Batch C headline) The headline binary demo. Unresolved binaries blend into one photometric source of inflated mass (Tout+1996 ZAMS L-M from fluxax), distorting the observed mass function. Recover the binary fraction f_b from the distortion, and show it requires the correct Moe & Di Stefano (2017) P-q coupling: - Forward model: mu_k(f_b) = N_sys[(1-f_b)S_k + f_b B_k] (linear in f_b); single template S_k (IMF) + binary template B_k (per-binary catalogue stars after resolution a<a_crit + Tout blending). Per-bin Poisson MLE + Fisher. - Controlled coupling test: the independent comparison shuffles q against P, so the q and P MARGINALS are identical and only the correlation differs -> any bias is the coupling alone. - Result (N_sys=3e5, ALL PASS): blended subset median q Moe 0.540 vs independent 0.479 (close binaries more equal-mass under Moe); Moe template recovers f_b 0.5016+-0.0074 (+0.22s); independent template biased 0.4749+-0.0070 (-3.6s, ~5% LOW). The wrong-model bias is a systematic -> significance grows as sqrt(N_sys) (-1.5s at 5e4, -3.6s at survey scale). fluxax (Tout ZAMS) added as a CI-safe, UNLOCKED experimental dep (private-sibling pattern, like jaxstroviz): documented in pyproject [experimental], guarded import, local 'uv pip install -e ../fluxax --no-deps'; uv.lock unchanged (lock-check green). Tout1996 bib entry. Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ification (Batch C) A paper-seed demo on velocity anisotropy via the beta(r) channel: - (a) WELL-SPECIFIED: a Plummer+OM cluster (true r_a=1.5); the closed-form OM profile beta=r^2/(r^2+r_a^2) fit recovers r_a (1.55+-0.035, +1.49s) with a good fit (reduced chi^2=0.43, conservative beta_se) and a sigma(r_a)~N^-1/2 forecast. - (b) MISSPECIFIED (the paper point): a Michie anisotropic-King cluster (true r_a=6) is a genuinely different DF (not OM's f(Q)). Fitting it with the OM form inflates the reduced chi^2 to 5.58 -- 12.9x the well-specified fit -- AND mis-estimates r_a (8.9 vs 6.0). The anisotropy formalism you assume shapes what you infer. Honest scope: no King+OM model exists (King's anisotropic form IS Michie), so the misspecification is an OM fit to a Michie SAMPLE on beta(r), not a fixed-density refit. beta channel via binned_sigma_beta + beta_se=(1+|beta|)/sqrt(n); fit via gaussian_loglike + mle_adam + fisher_cov. ALL PASS (2026-06-12, N=30000, ~9 s). Website page + myst TOC + run-record. Completes the 6 core Batch C demos (B9, B11, B10, B5, B4, B6). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Six gated-CLI science demos (B9/B11/B10/B5/B4/B6) + the Poisson number-density harness channel + fluxax CI-safe integration + docs Makefile. All ALL PASS, myst 157 pages 0 warnings. Awaits Anna for the ONE final PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Visual inspection of the rendered figures found the 3-sigma detection-N marker (N~57 in B5, N~148 in B6) sat clipped at the far-left edge because it fell BELOW the plotted N range [1e3, 3e5]. Extend the forecast x-grid down to 30 so the marker is in-range, and reposition the annotation (bottom, va=bottom) so it does not collide with the panel label. Both demos still ALL PASS; physics unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ety) A truncation fit (B7) has bins where mu==0 AND dmu/dz==0 beyond the edge: 1/mu is inf and 0*inf=NaN, so the Fisher came out NaN/singular. Floor mu with jnp.maximum exactly as poisson_loglike already does; the zero-Jacobian empty bins then contribute 0. New regression test (empty-mu bins -> finite F equal to the populated-bin closed form). 29/29 _demo_inference tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…C stretch) Reuses the Poisson number-density channel (B11) to recover an EFF young cluster's truncation radius r_t from the outer counts, then -> Galactocentric distance: - a is pinned by the inner profile (fixed); r_t read from the OUTSKIRTS. Bins extend past r_t so the outermost are empty -> the Poisson -mu term penalizes too-large r_t; too-small leaves outer stars unexplained. Shallow gamma=2.5 so stars reach r_t (a steep halo self-truncates, leaving r_t in the noise). - r_t made differentiable WITHOUT a hard jnp.where (zero gradient through the boundary): integrate the untruncated cumulative once, evaluate at bin edges CLIPPED at r_t, normalize by enc(r_t) -> real dmu/dr_t for a sharp truncation. - ALL PASS (N=2e4, ~5 s): r_t 12.0->12.01+-0.04 (+0.24s); 93% of the r_t Fisher info from the outer bins (spikes at the edge); sigma(r_t)~N^-1/2; Jacobi -> R_gal = 2.79+-0.01 kpc (round-trips through jacobi_radius). King1962 bib entry (tidal radius, AJ 67,471, verified from tidal.py). Website page + myst TOC + run-record. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…h C realism bridge) The first observational-realism demo. A solid-body-rotating cluster (apply_solid_body_rotation) is viewed at inclination i via a NEW line-of-sight projection helper (project_los: tilt + LOS); the mean line-of-sight velocity is linear in the sky x with slope k = omega*sin(i): - recover k from binned <v_los>(x_sky): 1.69+-0.03 (truth 1.732, pull -1.34); - DEGENERACY headline: <v_los> measures the PRODUCT omega*sin(i), not omega. The (omega,i) Fisher is RANK 1 (eigenvalues [8e-14, 1887], cond 2.3e16) -- every (omega,i) on omega=k/sin(i) fits equally; breaking it needs an independent inclination (the projected flattening) that LOS velocities cannot supply. - forecast sigma(k) ~ N^-1/2. The bridge to a future observational-realism arc (projection introduced here; measurement errors / selection / PSF are the rest, applied to all demos). ALL PASS (N=2e4, ~4 s). Website page + myst TOC + run-record. Completes Batch C: 6 core demos (B9/B11/B10/B5/B4/B6) + 2 stretch (B7/B8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

drannarosen and others added 16 commits June 12, 2026 18:45

docs(batch-c): STATUS — all 8 demos done (6 core + 2 stretch B7/B8)

15b6753

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

drannarosen merged commit 903df2b into main Jun 13, 2026
9 checks passed

drannarosen deleted the feat/batch-c-demos branch June 13, 2026 05:02

Navigation Menu

Repository navigation

Batch C — Science Demos II: 8 differentiable-inference demos + harness - #7

Batch C — Science Demos II: 8 differentiable-inference demos + harness#7
drannarosen merged 16 commits into
mainfrom
feat/batch-c-demos

drannarosen commented Jun 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

drannarosen commented Jun 13, 2026

Summary

Harness

Infrastructure / fixes

Two physics/harness catches (with regression tests)

Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant