[WIP] Space Charge: bound the dynamically resized mesh - #1621
Open
ax3l wants to merge 2 commits into
Open
Conversation
ax3l
force-pushed
the
topic-igf-greens-cache
branch
from
August 27, 2026 21:18
9275202 to
e4a7fee
Compare
ax3l
force-pushed
the
topic-igf-greens-cache
branch
3 times, most recently
from
August 27, 2026 22:02
593a164 to
f6c2fed
Compare
ResizeMesh fitted the box to the exact particle extent on every slice step, so the resolution drifted continuously and the FFT solver had to rebuild its Green's function each time. Give the padding a band instead: geometry.prob_relative stays the smallest the mesh may be and the new geometry.prob_relative_max the largest. Within that band the mesh is chosen from a fixed set of lengths, so a beam of nearly the same size lands on exactly the same mesh and the Green's function can be reused. The allowed lengths are 2^(k/m) meters, with m = ceil(1/log2(r)) for a band r = prob_relative_max/prob_relative[0]. Rounding m up keeps the mesh inside the band, and keeping it a whole number of lengths per doubling makes two meshes a factor of two apart differ by exactly two, which the solver relies on to reuse a Green's function built at another scale. The longitudinal axis is treated in the frame the solver works in, so a changing reference energy resizes the mesh by itself. Setting prob_relative_max equal to prob_relative[0] fits the beam exactly, which stays bit-identical to the previous behavior. Also consolidates the two diverging copies of read_mr_prob_relative, which had to be kept in sync by hand for every new geometry parameter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ax3l
force-pushed
the
topic-igf-greens-cache
branch
from
August 27, 2026 22:55
f6c2fed to
a86d0af
Compare
3 tasks
ax3l
added a commit
to AMReX-Codes/amrex
that referenced
this pull request
Aug 28, 2026
`OpenBCSolver` rebuilds its spectral Green's function whenever `setGreensFunction()` is called, and there is no way to hand it one that was built earlier. Add `greensFunctionFFT()` accessors so a caller can keep its own set and move them into and out of the solver with `std::swap` rather than recomputing. `solve()` only reads the spectral Green's function, and `prepare_openbc()` concerns the R2C plans rather than the Green's function values, so one moved in this way needs no further preparation. The one caveat, documented at the accessor: in 2D, `m_G_fft` is an alias of the Green's function R2C's spectral data, which a later `setGreensFunction()` writes into. Callers may write through the reference there, but must not swap or otherwise re-seat it. Additive only, so nothing changes for existing users. Exercised downstream in BLAST-WarpX/warpx#7178 & BLAST-ImpactX/impactx#1621, where reusing Green's functions this way reproduces rebuilding them bit for bit, and where it takes an accelerating beam simulation from 100 Green's function builds to 10. - [x] 🤖 Generated with [Claude Code](https://claude.com/claude-code) - [x] tested with ABLASTR/WarpX and ImpactX - [x] manually self-reviewed --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ResizeMeshfitted the box to the exact particle extent on every slice step, so the resolution drifted continuously and the FFT solver rebuilt its Green's function each time. This gives the padding a band instead, and within it picks the mesh from a geometric progression (also used in industry), so a beam of nearly the same size lands on exactly the same mesh and the Green's function can be reused. On the newexpanding-acctest that takes builds from 100 to 10 and runtime from 0.95 s to 0.41 s, and oncfchannelfrom 50 to 4.The allowed mesh lengths are$L_k = 2^{k/m}$ m, with $m = \lceil 1 / \log_2 r \rceil$ and $r$ the width of the band, $L_k$ is the mesh edge length along one axis, $k$ the whole number making $L_k$ the smallest such length that still covers the padded beam, and $m$ how many allowed lengths there are per factor of two. Rounding $m$ up keeps the mesh inside the band, and keeping it a whole number per doubling makes two meshes a factor of two apart differ by exactly two, which the solver relies on to reuse a Green's function built at another scale.
prob_relative_maxdivided byprob_relative[0]. Heregeometry.prob_relativeis now the smallest the mesh may be andgeometry.prob_relative_maxthe largest, defaulting to 10% above it for the FFT solver, soTesting detail
Adds
tests/python/test_igf_greens_cache.py. It compares the reusing solver againstablastr.igf_rebuild_always=1and requires exact equality of all beam moments, covering the store disabled and enabled, an exactly fitted mesh and a banded one, and an accelerating lattice. Each configuration runs in a single-threaded subprocess: charge deposition sums thread-dependently, so under the suite'sOMP_NUM_THREADS=2even two identical runs differ in the last digits and the comparison would otherwise be vacuous. Running apart also keeps ParmParse from leaking between configurations. Further tests assert the mesh is one of the allowed lengths for bands giving 4, 8 and 16 lengths per doubling, with a negative control asserting an exactly fitted mesh is not, and that a band belowprob_relative[0]is rejected.How wide the band can get, scanned on
expanding-accagainst its own envelope comparison atrtol0.79%: 1.21 gives 0.496% and 76 of 100 reused, 1.5 gives 0.554% and 90 reused, 2.0 gives 0.640% and 94 reused, 3.0 gives 1.115% and fails. So the default is conservative and roughly 2.0 is the ceiling for this test. Emittances are unchanged to 0.00% throughout.expanding-2d-accpasses at every band and reuses 91 of 100 at the default, though the Green's function is only about 2% of its runtime.Space-charge regressions: 66 of 66 pass, including both new acceleration tests. The 100-slice
expanding-acccase remains byte-identical to rebuilding every step while reusing 90 of 100, since the meshes recur exactly rather than approximately.Also consolidates the two diverging copies of
read_mr_prob_relative, inimpactx::initialization::andimpactx::detail::, which had to be kept in sync by hand for every new geometry parameter.🤖 Generated with Claude Code