Skip to content

[WIP] Space Charge: bound the dynamically resized mesh - #1621

Open
ax3l wants to merge 2 commits into
BLAST-ImpactX:developmentfrom
ax3l:topic-igf-greens-cache
Open

[WIP] Space Charge: bound the dynamically resized mesh#1621
ax3l wants to merge 2 commits into
BLAST-ImpactX:developmentfrom
ax3l:topic-igf-greens-cache

Conversation

@ax3l

@ax3l ax3l commented Aug 19, 2026

Copy link
Copy Markdown
Member

ResizeMesh fitted the box to the exact particle extent on every slice step, so the resolution drifted continuously and the FFT solver rebuilt its Green's function each time. This gives the padding a band instead, and within it picks the mesh from a geometric progression (also used in industry), so a beam of nearly the same size lands on exactly the same mesh and the Green's function can be reused. On the new expanding-acc test that takes builds from 100 to 10 and runtime from 0.95 s to 0.41 s, and on cfchannel from 50 to 4.

The allowed mesh lengths are $L_k = 2^{k/m}$ m, with $m = \lceil 1 / \log_2 r \rceil$ and $r$ the width of the band, prob_relative_max divided by prob_relative[0]. Here $L_k$ is the mesh edge length along one axis, $k$ the whole number making $L_k$ the smallest such length that still covers the padded beam, and $m$ how many allowed lengths there are per factor of two. Rounding $m$ up keeps the mesh inside the band, and keeping it a whole number per doubling makes two meshes a factor of two apart differ by exactly two, which the solver relies on to reuse a Green's function built at another scale.

geometry.prob_relative is now the smallest the mesh may be and geometry.prob_relative_max the largest, defaulting to 10% above it for the FFT solver, so $m = 8$. Setting them equal fits the beam exactly and is bit-identical to before. The longitudinal axis is sized in the frame the solver works in, and that stretch is derived here the same way the solver derives it, because reconstructing it from a velocity costs about $\varepsilon\gamma^2$ and would leave the mesh never quite repeating. Depends on BLAST-WarpX/warpx#7178, which is what actually reuses the Green's function.

Testing detail

Adds tests/python/test_igf_greens_cache.py. It compares the reusing solver against ablastr.igf_rebuild_always=1 and requires exact equality of all beam moments, covering the store disabled and enabled, an exactly fitted mesh and a banded one, and an accelerating lattice. Each configuration runs in a single-threaded subprocess: charge deposition sums thread-dependently, so under the suite's OMP_NUM_THREADS=2 even two identical runs differ in the last digits and the comparison would otherwise be vacuous. Running apart also keeps ParmParse from leaking between configurations. Further tests assert the mesh is one of the allowed lengths for bands giving 4, 8 and 16 lengths per doubling, with a negative control asserting an exactly fitted mesh is not, and that a band below prob_relative[0] is rejected.

How wide the band can get, scanned on expanding-acc against its own envelope comparison at rtol 0.79%: 1.21 gives 0.496% and 76 of 100 reused, 1.5 gives 0.554% and 90 reused, 2.0 gives 0.640% and 94 reused, 3.0 gives 1.115% and fails. So the default is conservative and roughly 2.0 is the ceiling for this test. Emittances are unchanged to 0.00% throughout. expanding-2d-acc passes at every band and reuses 91 of 100 at the default, though the Green's function is only about 2% of its runtime.

Space-charge regressions: 66 of 66 pass, including both new acceleration tests. The 100-slice expanding-acc case remains byte-identical to rebuilding every step while reusing 90 of 100, since the meshes recur exactly rather than approximately.

Also consolidates the two diverging copies of read_mr_prob_relative, in impactx::initialization:: and impactx::detail::, which had to be kept in sync by hand for every new geometry parameter.

🤖 Generated with Claude Code

@codspeed-hq

codspeed-hq Bot commented Aug 19, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 70 untouched benchmarks


Comparing ax3l:topic-igf-greens-cache (f5f22e1) with development (2a2c2a7)

Open in CodSpeed

@ax3l
ax3l force-pushed the topic-igf-greens-cache branch from 9275202 to e4a7fee Compare August 27, 2026 21:18
@ax3l ax3l changed the title [WIP] Space Charge: quantize the dynamically resized mesh [WIP] Space Charge: bound the dynamically resized mesh Aug 27, 2026
@ax3l
ax3l force-pushed the topic-igf-greens-cache branch 3 times, most recently from 593a164 to f6c2fed Compare August 27, 2026 22:02
ResizeMesh fitted the box to the exact particle extent on every slice
step, so the resolution drifted continuously and the FFT solver had to
rebuild its Green's function each time. Give the padding a band instead:
geometry.prob_relative stays the smallest the mesh may be and the new
geometry.prob_relative_max the largest. Within that band the mesh is
chosen from a fixed set of lengths, so a beam of nearly the same size
lands on exactly the same mesh and the Green's function can be reused.

The allowed lengths are 2^(k/m) meters, with m = ceil(1/log2(r)) for a
band r = prob_relative_max/prob_relative[0]. Rounding m up keeps the
mesh inside the band, and keeping it a whole number of lengths per
doubling makes two meshes a factor of two apart differ by exactly two,
which the solver relies on to reuse a Green's function built at another
scale. The longitudinal axis is treated in the frame the solver works
in, so a changing reference energy resizes the mesh by itself. Setting
prob_relative_max equal to prob_relative[0] fits the beam exactly, which
stays bit-identical to the previous behavior.

Also consolidates the two diverging copies of read_mr_prob_relative,
which had to be kept in sync by hand for every new geometry parameter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ax3l
ax3l force-pushed the topic-igf-greens-cache branch from f6c2fed to a86d0af Compare August 27, 2026 22:55
ax3l added a commit to AMReX-Codes/amrex that referenced this pull request Aug 28, 2026
`OpenBCSolver` rebuilds its spectral Green's function whenever
`setGreensFunction()` is called, and there is no way to hand it one that
was built earlier. Add `greensFunctionFFT()` accessors so a caller can
keep its own set and move them into and out of the solver with
`std::swap` rather than recomputing. `solve()` only reads the spectral
Green's function, and `prepare_openbc()` concerns the R2C plans rather
than the Green's function values, so one moved in this way needs no
further preparation.

The one caveat, documented at the accessor: in 2D, `m_G_fft` is an alias
of the Green's function R2C's spectral data, which a later
`setGreensFunction()` writes into. Callers may write through the
reference there, but must not swap or otherwise re-seat it.

Additive only, so nothing changes for existing users. Exercised
downstream in BLAST-WarpX/warpx#7178 &
BLAST-ImpactX/impactx#1621, where reusing
Green's functions this way reproduces rebuilding them bit for bit, and
where it takes an accelerating beam simulation from 100 Green's function
builds to 10.

- [x] 🤖 Generated with [Claude Code](https://claude.com/claude-code)
- [x] tested with ABLASTR/WarpX and ImpactX
- [x] manually self-reviewed

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant