Geometry of the cosmic web: research program, results, and two posts#19
Open
moiseevigor wants to merge 49 commits into
Open
Geometry of the cosmic web: research program, results, and two posts#19moiseevigor wants to merge 49 commits into
moiseevigor wants to merge 49 commits into
Conversation
Unify terminology, notation, and numeric constants across Parts 1–2 and
appendices A1–A5 so the prose, the math, and the computed figures agree.
Review-pass corrections in this change:
- Figure-eight / first-Maxwell modulus set to k ≈ 0.909 (the root of
2E(k²)=K(k²)), matching the generator/data, in Part 2 and A5
(previously 0.71 in Part 2 and 0.84/0.85 in A5).
- Abnormality constant made consistent: Part 2 now uses ν∈{0,1} with
normal ν=1, matching the PMP derivation in appendix A3.
- A5 non-inflectional spatial period corrected to 2K(m) (was 2√m·K(m),
which is the pendulum-time rotation period); now agrees with Part 2.
- A5 Fig A5.2: the σ-pair first Maxwell time is stated and computed as
4K(k²) for all k; the JS uses the analytic value since y_A = 2k(1−cn)
is ≥ 0 and only touches zero, so the old sign-change search could miss it.
- A5 Fig A5.3: reference modulus (→0.909) and cusp-onset (2π = 4K(0),
the smallest period in the swept family) corrected.
- A4: asymptotic-period comment 16→32 to match the code (T ~ 2 log(32/(1−E))).
…nt geometry Scoping document (unpublished post) for testing whether the cosmic web is better extracted — and possibly shaped — by sub-Riemannian geodesics on R^3 x S^2 = SE(3)/SO(2). Defines claims C1/C2, hypotheses H1-H4 with kill criteria, experiments E0-E3 + theory item T1 on public data, metrics M1-M6, caveats, glossary, and references. Grew out of a deep-research thread on universe filaments.
…d-length benchmark Modules: synthetic web generators (Voronoi filament model with exact spine/junction truth, optional Bezier bending; Zel'dovich box), orientation score + hypoelliptic diffusion on R^3 x S^2, Hessian-baseline ridgeness, matched-length skeleton extraction with shared post-processing, M1/M2 metrics. Calibrate-on-seed-1 / evaluate-held-out protocol in run_e0.py, assert-based smoke tests.
…ss, diffusion rejected Curved-filament variant (Bezier-bent Voronoi edges), 5 held-out seeds, per-method calibration on seed 1. Findings: SE(3) orientation lift beats the Hessian baseline on spine completeness at sparse sampling (+0.067 straight / +0.064 curved at n_gal=2500, 5/5 seeds each), ties junctions, slightly loses at dense sampling; hypoelliptic diffusion rejected by calibration at both curvature levels — gains come from elongated oriented filters. Report with figures in docs/E0-report.md.
…riant Parallel eval (--eval-seeds N, 8-worker pool; 200 realizations per variant in ~150 s). Wilcoxon signed-rank on paired per-seed deltas added to the report. Sparse-regime completeness gain confirmed at scale: +0.070 (48/50 seeds, p=6e-14) straight, +0.069 (49/50, p=1e-14) curved; dense regime deficit -0.02 also significant. Verdict unchanged, now powered.
…ables, per-level Wilcoxon Delta box plots with per-seed points, median/p10-p90 quantiles, and per-level significance for both dC and djF1. The n=50 data resolves the junction question: significant crossover (lift wins jF1 at n_gal=2500, Hessian at dense levels), verdict updated accordingly.
…on prediction refuted Sweep A: hypoelliptic diffusion (splitting scheme) hurts completeness at every bend x sparsity combination, including the most-curved tercile (-0.04..-0.25, p~2e-06) — the lift's value is the angularly-sharp measurement, not evidence propagation, at least with this numerical scheme. Sweep B: with node_frac=0 the Hessian wins junction F1 slightly (-0.03, p=0.014-0.064) while the lift keeps +0.05 completeness; the earlier sparse-junction win came from node clumps, not continuity.
… alignment win 2x2 clean-reference design reveals method-affinity (~0.1C) that dwarfs the method effect: H1 transfer on gravity fields is inconclusive via skeleton-vs-skeleton scoring. Tidal weighting (H2 first pass) null to negative. Clean finding: unweighted lift spines align with the tidal eigenframe at 0.73-0.76 vs Hessian 0.67 (null 0.5) with no tidal input. Also: truncated ZA option, adaptive smoothing, skeleton-length-corrected matched extraction (fixes hysteresis flooding on connected webs).
… PM N-body; E1b script Minimal EdS particle-mesh N-body (KDK leapfrog, spectral Poisson) validated on growth + collapse. E2 on 4 sims, 50k tracked particles each: ballistic corrections are perpendicular to the filament axis in the bulk (P2 = 0.21-0.23 vs null 1/3 — transverse pancake infall), weakly parallel only within ~4 vox of spines (0.364 +/- 0.010). Velocity-axis alignment flat at ~0.56. H4 as a transport principle is refuted for assembly; survives only as weak longitudinal streaming inside formed filaments. Also: tidal_frame moved to fields.py; E1b (method-neutral mass-coverage H1 + H2 upper bound) added.
…hypotheses exhausted
Band coverage: Hessian wins filament-band mass too (+0.018/+0.043,
p=0.0005); sig_par in {3,4.5,6} never closes the gap — the gravity-field
loss is genuine, not scale mismatch. SYNTHESIS.md records the full chain:
H1 sparse-toy win does not transfer to gravity webs; H2 dead; H4 refuted
in bulk (transverse pancake infall), weak longitudinal residual in tubes;
lifted geometry survives as static descriptor (M3). Scoping post gets a
status callout. E3/H3 (real survey data) remains open.
…s + E1b on N-body New unpublished post 'Two Ways to See a Cosmic Filament': both models with full math, fairness protocol, and five interactive D3 figures built from the real benchmark data (slice explorer across sampling levels, crossover chart, 50-seed paired-delta strip plot, ZA/PM band-coverage bars, E2 dynamics) — all verified rendering in a browser harness. E1b-pm: on real N-body fields the short-cigar lift ties the Hessian at sparse sampling (p=0.55) but never leads; failure-3 section updated with the nuance. Assets baked from result JSONs by make_post_assets.py.
…ngth artifact The matched-length rule was enforced on mask volume, not the realized skeleton; lift skeletons ran ~19% longer at sparse levels, buying the +0.07 completeness 'win' (p~1e-14). Corrected extractor (skeleton-length iteration) flips the verdict: Hessian matches or beats the lift at every sampling level on both toy variants (50/50 seeds, p~1e-15 from n_gal=2500 up), tie only at ultra-sparse 1200. E0/E0b rerun (50 seeds, +1200 level), E0c hybrid re-referenced (hybrid wins purity/junctions at 5k, not completeness), E1b-256 resolution check added (Hessian lead survives resolved cross-sections). Post, post assets, synthesis, and scoping-post status all rewritten to the corrected verdict; the post now tells the artifact story as its centerpiece lesson.
…sted Splitting-convergence test closes the diffusion caveat (Trotter-fine == coarse; operator-level verdict). Sparsity-limit sweep: no honest lift-win regime exists — Hessian wins again below n_gal=800 (p=0.003); the 800-1200 tie is a narrow band. Gravity hybrid halves the deficit but still trails. Post decision table and diffusion caveat updated; synthesis addendum.
…pines; program complete BOSS CMASS-N (z 0.45-0.55, 274k galaxies, 18 tiles of 512 Mpc/h) x Planck. H3 supported: both methods' spine networks carry a ~4-5 sigma Compton-y excess over footprint-matched controls (lensing null, noise-dominated); Hessian >= lift on the cross-signal at matched length (4.8 vs 4.1 sigma), consistent with corrected H1/E1b. M3-real: the lift's tidal-alignment advantage replicates on the sky (0.677+/-0.016 vs 0.617+/-0.016 over 18 tiles). Caveats logged (control latitude, tracer-halo gas, single shell). Pipeline: catalogue -> comoving tiles -> matched-length spines -> healpy stacking. Post gains a real-Universe coda; synthesis and scoping status updated. All hypotheses H1-H4 now carry empirical verdicts.
…achable, no bug language Fishing-net metaphor for the length mismatch, 'level the race' framing throughout, lesson restated as 'measure the thing that actually wins races'. Content and numbers unchanged.
…rict nulls; mostly halo gas Latitude-matched nulls raise the y-stack to 8.99/6.86 sigma (H3a supported, foreground-robust). Tracer-halo masking leaves 1.95/0.07 sigma (H3b: between-halos WHIM is a hint, not a result, at Planck depth). Radial profile extended to >=60 arcmin but non-discriminating at a 10 arcmin beam (H3c). Hessian > lift on every cross-signal metric. Program closed: deeper tSZ data (ACT/SPT) is the only way forward on H3b.
…t animations Math: kramdown was eating underscores inside inline $...$ spans before KaTeX ran — converted all 130 inline spans to kramdown-protected $$...$$. Charts: the global *.json exclude silently dropped post_data.json; data now ships as public/data/cosmic-web/post_data.js (window.CW_DATA), matching the repo's existing v1_geodesics.js pattern; fetch replaced by script src. All six interactive figures verified against the served HTML in a browser. New: rotating-cigar angular-selectivity widget (results post) and anisotropic-collapse animation (scoping post), plus 'in plain words' interludes after the heaviest formulas.
…ter conversion
The bulk $->$$ conversion had also run inside the results post's script
block, corrupting JS template literals (${ -> $${): tooltips printed
'$Hessian: $0.334' and SVG transforms went invalid, hiding axes, labels,
and the legend. Repaired all 10 literals and protected script/style blocks
from future text passes. Also fixed the mis-paired multi-line math the
converter left in the M4/M5 metric bullets and two glossary entries, and
merged 23 line-start inline spans that kramdown would mis-parse as display
math. Verified in-browser on the served pages: zero katex-error elements
on both posts, all 6 interactive figures with full axes/legends.
…d bare 'LI' Appendices A1-A5 get 'listed: false': filtered from the home page, archives, atom feed, tag pages, and collated lists via a shared where_exp filter (index wraps the paginator card in unless). Their URLs stay live, reachable only through the series navigation inside the posts. Also expanded three bare 'LI' abbreviations to 'left-invariant' (A1, A3) — the shorthand was never defined anywhere in the series.
…d pass Pagination fix: A1-A5 move from _posts to an _appendices collection with pinned permalinks — out of site.posts entirely, so the paginator, archives, and feeds are naturally clean (home page shows all 5 slots again); URLs unchanged, series nav untouched. Proofread fixes across A1-A5: first-use expansions (V1, SR, PMP, ODE, BCH, AGM, E-L) per file; math corrections — A3 dual basis and structure constants stated in the X-frame, h3-at-cusps wording, c := h1^2+h2^2 defined, energy-vs-Casimir cylinder disambiguated; A5 bi-invariant-metric claim corrected to agree with A1, phi0->phi slip, conjugate-time inequality direction, W_T range, 39-trajectory count; A4 AGM iteration arithmetic, three section cross-references, placeholder code replaced with a runnable event-based period finder; option-tag TeX to plain text; hyphenation and multi-underscore span hardening. Also stages run_e3c.py (ACT pipeline, pending data).
… the gate Masked residuals 1.77/1.11 sigma (3'/7' masks): halo-only model survives; upper limit, not detection. Key diagnosis: only 45% footprint overlap and unmasked stack at 2.95 sigma (vs 9 on Planck) — the shell-projected signal lives at degree scales that the ACT DR6 ILC filters out; ACT-depth WHIM work needs thin-shell or single-filament designs. Pre-registered gate (>=3 sigma to proceed) fails -> observational branch closes honestly.
…kage-bounded), ACT clean but underpowered 570k CMASS pairs (6-14 Mpc/h transverse, <6 LOS), halo-symmetric estimator: Planck bridge 3.39e-8 +/- 0.23e-8 (15 sigma, includes second-halo beam leakage -> upper bound); ACT 8.2e-9 +/- 9.4e-9 (0.87 sigma, leakage-free but underpowered for a 1e-8 bridge). Consistent with Model B at the 1e-8 level. E3d-v2 spec'd: full z-range, disc sampling, ring controls.
…T; Model B established Powered pair design (876k pairs z 0.43-0.70, 2-arcmin disc sampling, +/-60/90/120-degree ring controls): ACT DR6 bridge 2.28e-8 +/- 0.43e-8 (5.24 sigma, leakage-free), Planck 2.97e-8 (19.3 sigma) — consistent amplitudes across instruments. The halo-only model is rejected by an estimator that nulls symmetric halos by construction. Gate (>=3 sigma) passed; publication-grade caveats listed. Program closes on a positive terminal measurement produced by E3c's own diagnosis.
…to its honest core Jackknife (50 RA patches) deflates SNRs 1.6-2.4x: ACT on-axis 3.30 sigma. Unconnected-pair nulls: zero on ACT, 4.7 sigma fake bridge on Planck (1.8e-8) — beam leakage confirmed quantitatively. Null-subtracted bridge: 1.4e-8 (ACT, 1.6 sigma) / 1.2e-8 (Planck, 2.2 sigma) — reproduces the published amplitude; a >3 sigma bridge-specific claim needs model-based halo subtraction + joint likelihood (paper scope). Post coda and scoping status downgraded from the 5.2-sigma headline to the validated statement. Separation bins flat within errors.
…ect tone Headings lose manual numbers (the distill layout auto-numbers h2, which produced '5. 4. Experiments'); section cross-references now use names. 40 kramdown footnotes added across both posts defining jargon at first use (Lambda-CDM, FLRW, hypoelliptic, jackknife, Compton-y, ...), verified rendering with zero KaTeX errors. Collapse widget recentred (540px viewBox) and both posts get responsive figure margins — no horizontal overflow at 375px. Results post rewritten to direct language: subtitle, description, callout, the evaluation-flaw section (fisherman metaphor dropped), delta-figure lead-in, and the closing summary.
…tively; program table complete In D-time the Zel'dovich flow is free motion (potential absorbed into initial velocities; nothing to conformalize), and adhesion's exact variational principle is Hopf-Lax: quadratic-cost optimal transport on a flat metric (Brenier/MAK). Filaments are shocks of the transport map, not geodesics of a lifted metric; the derivation predicts both E2 signatures and explains the lift's descriptor-not-transport role. Last open item of the original scoping table closed.
…orrection is transverse Residual between true PM transport and per-particle ZA baseline: large (RMS 4.4 vox near spines), tidal-frame organized, but pointing ACROSS the filament axis at every distance ((R.e3)^2 = 0.21-0.30 < 1/3; Rpar/Rperp = 0.63-0.89). The adjustment term current models need is anisotropic transverse damping at shell-crossing (adhesion-like), not longitudinal geodesic transport. H4' refuted as stated; constructive form of the true correction measured. pm.py exposes per-particle ZA baselines.
…A and sticking at the web Effective model (ZA rays + tidal-frame transverse momentum damping at shell-crossing) vs PM truth, competitor-favouring calibration, 5 held-out seeds: near-web median error -7% vs ZA and -14% vs isotropic sticking; best overall (4.91 vs 5.00 vs 5.34); small +4% deficit vs ZA at 2-4 vox (premature damping from noisy frames). H-T2 supported where the physics predicts; refinements left as tuning. The adjustment-term conjecture is now a working model term.
…cs uniformly With tidal frames from the true final field, the transverse-damping model beats ZA in every bin (-20% at the web, -10% overall; the 2-4 vox deficit closes and flips). The E5 deficit was frame-estimation error. Oracle bound 4.47 vs proxy 4.93 prices the recoverable error and makes frame estimation the top refinement lever.
…ta=0.75, -7% vs ZA held-out Selection over 6 configs (frames source, beta, rho_c, sequential damping), top-2 validated on held-out seeds: winner recovers ~63% of the E5b oracle gap (4.64 vs proxy 4.93, bound 4.47). Partial damping unlocks self-density frames (full damping over-triggers via density feedback); pancake-ordered sequential damping underperforms at this resolution.
…; model card Final grid: beta=0.6, frame smoothing 2 Mpc/h wins held-out at 4.52 vs ZA 4.98 (-9%), 0.05 vox from the E5b oracle bound — 89% of recoverable error closed; further knobs are below the noise floor. MODEL-CARD.md documents the frozen recipe: capabilities, limitations, per-segment table, failure modes (over-damping feedback, outskirt frames, post-crossing physics), integration guide, and reproduction commands.
…th four new result figures Post now follows conjecture -> refutation-with-validity-batteries -> correction measured (E4) -> correction turned into a model (E5) -> oracle vindication (E5b) -> optimization to the bound (E5c/d) -> model card. New sections 'The correction, measured' and 'From correction to model, to the bound', plus four D3 figures built from the frozen experiment results: claims-under-controls SNR ladder (bridge signal under stricter controls, both instruments), E4 residual-direction chart with the isotropic null, the model ladder against the 4.47 oracle bound, and per-distance-bin gains (ZA vs refined vs oracle). All nine interactives verified structurally on the served page; zero KaTeX errors; model card linked.
…th clustering Zero-recalibration transfer battery (2 Mpc voxels; sigma8 0.6/1.0): advantage over ZA persists everywhere and scales with nonlinearity (-3%/-9%/-13% overall, up to -22% at the web at sigma8=1) — physical trend, not tuning artifact. Model card scope updated.
l-middle figures no longer span the 1100px golden-ratio width; they now match the text container exactly (--body-w + gap + --gutter-w). E4 chart's series labels moved from clipped line-ends to an in-plot legend. Verified: every figure measures 932px, equal to .l-body.
… scope complete Frozen model under flat LCDM (Om=0.31): -10%/-17% overall/web, matching EdS; at 0.5 Mpc voxels: -10%/-15% (single seed), with near-identical physical errors across all regimes — cosmology- and resolution-invariant, as a D-parametrized geometric correction must be. pm.py generalized to LCDM (growth factor, generic da/dtau stepping, Omega_m in Poisson). No testable scope rows remain.
…terion verified r(k)/T(k) vs PM truth: damping model beats ZA in the nonlinear regime on both statistics (r 0.39 vs 0.29 at k=0.53; T 0.41 vs 0.23 at k=0.34), extending usable k-range; isotropic sticking catastrophic. Caveats: ~7% large-scale amplitude deficit (rescalable), small mid-k phase cost. High-res row verified at n=3.
…frames to mechanism 2LPT and MUSCLE implemented from scratch, identical ICs: 2LPT degrades transport (8.07, shell-crossing overshoot); MUSCLE ties the frozen model (4.49+/-0.12 vs 4.52+/-0.18), wins small-scale r(k); damping model wins mid-scale T(k). Paper framing: E4's transverse-correction measurement explains why MUSCLE-class corrections work; single directional ingredient reproduces MUSCLE-level accuracy. E10: truth step-converged (0.008 vox); comparison invariant. Referee must-fix battery complete.
…E12 external validation launched Hybrid (MUSCLE collapse-time trigger + directional damping) worse than both parents (4.67 vs 4.49/4.52): the Eulerian self-density trigger carries information the precomputed Lagrangian time lacks; MUSCLE's edge lives in its divergence construction. CAMELS CV_0 data (2LPT ICs + Arepo z=0 snapshot, 370MB) downloaded and verified; E12 external-truth validation script added.
…ourced 'The correction beyond Zel'dovich is transverse: a measured mechanism for shell-crossing models of cosmic-web transport.' ~4,400 words of prose: quantitative abstract; intro situating ZA/2LPT/adhesion/MUSCLE/ALPT/COLA; methods with truth convergence and the frozen recipe; results tables (E4 residual anisotropy, model ladder vs oracle bound with MUSCLE tie and 2LPT degradation, frozen-knob transfer matrix, field-level fidelity); Hopf-Lax interpretation; secondary results (methodology lesson, finder comparison, compressed observational validation); honest limitations. Every number traced to a report; zero unsourced figures.
…paper updated Evolved from CAMELS CV_0 ICs, our PM matches the official Arepo z=0 positions to 0.41 Mpc/h median (matched IDs), r=0.999/0.984/0.937 at k=0.35/0.66/1.26 — near-perfect over all scales used by the program's claims. Frozen model does not invert even at 10x resolution extrapolation (1.127 vs ZA 1.140). Paper limitations and references updated; referee must-fix battery fully closed.
Full number audit against all reports: three precision corrections (E0 seed-count phrasing, r>=0.93 at k<=1.26, PM degradation onset k~2.4); E11 hybrid negative integrated into conclusions; notation normalized to h^-1 Mpc / h Mpc^-1 throughout; 5,475 words. Both cosmic-web posts set published: true (listings verified on the dev server; public go-live remains the merge to master). E12 parametrized for multiple suites.
…cond-code check to Astrid SIMBA_DM CV_0 turned out byte-identical to IllustrisTNG_DM CV_0 on the server (shared DM-only realization) — its E12 run added no information and its outputs are removed. Astrid_DM CV_0 (MP-Gadget, verified distinct bytes) downloaded as the genuine second external code; E12-astrid running.
…P-Gadget consensus at 0.029 Mpc/h Astrid (MP-Gadget) truth reproduces the Arepo verdict to three decimals; the codes agree with each other to 0.029 Mpc/h median per matched ID — 14x tighter than our PM's 0.41 offset from either, itself 10x below the measured effects. Shared-ICs premise verified per-ID (0.068 vox at z=15). Paper limitation upgraded to the two-code statement.
…ion Makefile, citation metadata Four publication figures (residual direction with isotropic null, model ladder against the oracle bound incl. 2LPT/MUSCLE, frozen-knob transfer bars, field-level r/T) generated from artifacts into docs/paper-figures/. requirements.txt pinned; Makefile maps every paper table/figure to its reproduction command; CITATION.cff added (MIT, per repo license).
main.tex (334 lines): article class + booktabs tables 1-4, the four figures with self-contained captions, manual bibliography (16 cited + 12 listed), all unicode converted, environments/braces/refs statically verified. Compilation deferred (no TeX engine on this machine).
Rework both posts as science communication for a general audience: plain-language narrative in the main text, technical machinery folded into five new appendices (B1 transport models, B2 tidal frame, B3 honest benchmarks, B4 transverse-damping recipe, B5 reading gas maps) wired into a series nav via _data/series.yml. - New 'three levels of reality' figure (BOSS wedge / PM simulation / Voronoi toy) showing how each synthetic dataset models the real sky - Model-ladder widget relabeled in plain words, MUSCLE rung added, oracle line renamed 'perfect-steering limit' - Heavy formulas demoted to footnotes (fixes kramdown table-mangling of piped math in footnotes with \vert) - 'In plain words' callouts at the evaluation-flaw verdict; appendix pointers at every deep-dive site - Research-program post: status callout rewritten as plain-language outcomes with links into Part 2 and appendices - distill.css: footnote/endnote styling + section-counter scope fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A complete research program (~60 commits) on cosmic-web filament geometry, run start to finish in this branch:
_appendices/, B1–B5) holding the full machinery — transport models, the tidal frame, honest benchmarking, the frozen transverse-damping recipe, reading gas maps — wired to the posts by a "Geometry of the Cosmic Web" series nav (_data/series.yml).research/cosmic-web/with 20+ experiment reports, SYNTHESIS.md, MODEL-CARD.md, an audited paper draft + arXiv-ready LaTeX + publication figures, pinned requirements, and a Makefile mapping every paper artefact to its reproduction command.Merging publishes the two posts and the five appendix pages on the public site.
🤖 Generated with Claude Code