Skip to content

Apply quadrature weights out of place so autograd survives GaussLegendre - #262

Open
gomezzz wants to merge 2 commits into
developfrom
fix/evaluate-integrand-inplace-weights
Open

Apply quadrature weights out of place so autograd survives GaussLegendre#262
gomezzz wants to merge 2 commits into
developfrom
fix/evaluate-integrand-inplace-weights

Conversation

@gomezzz

@gomezzz gomezzz commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

BaseIntegrator.evaluate_integrand ended with result *= weights — an in-place
multiply on the tensor the user's integrand had just returned. torchquad does not
own that tensor.

  • Breaks autograd on the headline feature. Any integrand whose last operation
    reads its own output in the backward pass — exp, sqrt, tanh, sigmoid,
    div, pow — raises RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation ... is at version 1; expected version 0. Same failure for gradients with respect to the integration
    domain, on CPU and GPU. Only the Gaussian family passes weights here
    (GridIntegrator._weights returns None), so Newton–Cotes and Monte Carlo were
    unaffected, as was JAX. Present since 0.4.0 (Gaussian quadrature #141).
  • Why nothing caught it: gradient_test.py listed six integrators but only
    five point counts, so zip silently dropped GaussLegendre — the only
    integrator that reaches this code — from every gradient test. And every existing
    test integrand (2|x|, polynomials) has a backward pass that does not read its
    own output, so even with GaussLegendre restored they would all still pass.

Found while validating the 0.6.0 release; not a 0.6.0 regression. It fails
loudly rather than returning wrong numbers, so no published result is at risk.

Related to #258.

Numerical behaviour

The multiplication is identical; only its destination changed. No tolerance was
loosened, and the full suite passes unchanged on all four backends.

One deliberate behaviour change: an in-place multiply takes its left operand's
dtype, so an integrand returning float32 under float64 precision silently
discarded the weights' precision. Out-of-place promotes instead, which is what
"no silent precision downgrades" requires. tests/base_integrator_test.py::test_weight_dtype_*
pins this.

GaussLegendre gets 499 rather than 149 points in the 1-D gradient tests: the
V-shaped integrands there have a kink, and Gauss–Legendre is only O(N^-2) on
|x| while being spectrally accurate on the smooth exponential at any of these
N. Raising N rather than loosening a tolerance.

Test plan

  • New tests/base_integrator_test.py — 10 tests asserting the integrand's
    returned tensor is unchanged after evaluate_integrand, for scalar and
    multi-value integrands, on numpy/torch/jax/tensorflow, plus the dtype guard.
    Verified these fail without the fix (numpy and torch; jax and tensorflow
    pass trivially since their tensors are immutable).
  • gradient_test.pyGaussLegendre restored to the matrix with a
    length assertion, and a new exponential integrand whose exact gradient is
    1.0. Verified this fails without the fix with the precise
    ExpBackward0 error, and passes with it, on torch/jax/tensorflow.
  • Full suite: 203 passed across all four backends (was 193; +10 new).
  • Gates: ruff check clean, ruff format --check clean (64 files),
    pydoclint torchquad/ no violations, vulture --min-confidence 100 clean.
  • Fix confirmed against an analytic reference: d/da ∫₀² e^{-a x²} dx at
    a = 0.7 gives -0.6561422331326556 versus a fine-grid reference of
    -0.6561422331326168.

What changed after review

  • The silent upcast now warns. Applying the weights out of place stopped the
    silent downcast but replaced it with a silent promotion, which the fail-hard
    rule objects to equally. A dtype mismatch now warns, matching the
    backend-mismatch warning ten lines above it. The check compares mantissa width,
    so a complex128 integrand against float64 weights stays quiet — it loses no
    precision — while float32 or complex64 against float64 does not. Verified
    against the full suite: the only warnings emitted are the four pre-existing
    Boole/Simpson N-adjustment ones.
  • The guarantee moved into the docstring. evaluate_integrand is public API
    and the tutorial shows users calling it directly; a contract that lives only in
    an inline comment is one refactor from being reintroduced as *=.
  • The parallel lists in gradient_test.py became one list of tuples, so they
    cannot drift out of sync again rather than being caught by an assert after the
    fact.
  • The exponential case skips MonteCarlo and VEGAS, which never reach the
    weights branch and were already covered by the four preceding integrands, and
    its bound tightened from 5e-2 to 1e-4. The old bound was set by Monte Carlo
    and would have passed a several-percent error in weight application on the
    deterministic rules; 1e-4 is limited by Trapezoid at 1.7e-5, while Boole and
    GaussLegendre reach 1e-15.
  • Two new tests: the mismatch warning fires, and a matching (or complex)
    integrand stays quiet.
  • Full suite now 205 passed, gates all still clean.

Not addressed here

Gaussian._weights(self, N, dim, backend, requires_grad=False) never receives
requires_grad from any of its four callers, so weights.requires_grad = requires_grad is a permanent no-op. Confirmed, but unrelated to this fix and
better as its own change — _roots does use the parameter, so it is not a
blanket removal.

evaluate_integrand ended with `result *= weights`, mutating the tensor the
user's integrand had just returned. That broke PyTorch autograd through
GaussLegendre for any integrand whose last operation reads its own output in
the backward pass (exp, sqrt, tanh, sigmoid, div, pow), raising "one of the
variables needed for gradient computation has been modified by an inplace
operation". It failed the same way for gradients with respect to the
integration domain, on CPU and GPU.

Only the Gaussian family passes weights through this path, so Newton-Cotes and
Monte Carlo were unaffected, as was JAX. Present since 0.4.0.

The multiplication is unchanged; only its destination is. As a side effect the
weights are no longer downcast to the integrand's dtype, so a float32 integrand
under float64 precision now keeps the weights' precision.

gradient_test.py listed six integrators but only five point counts, so zip
silently dropped GaussLegendre -- the only integrator that reaches this code --
from every gradient test. The lists are now length-checked and GaussLegendre
gets more 1D points, since the V-shaped test integrands have a kink and
Gauss-Legendre is only O(N^-2) there. A smooth exponential integrand is added
because every existing test function has a backward pass that does not read its
own output, and so could never have caught this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YP9oDLmw636HhPSAD6Nv3p
@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown

Overall Coverage

Coverage Report
FileStmtsMissCoverMissing
torchquad
   __init__.py26292%69–70
torchquad/integration
   base_integrator.py54787%41, 95–98, 102, 152, 156, 161
   boole.py30390%54, 73–76
   gaussian.py56296%121, 143
   grid_integrator.py97397%50, 164, 268
   integration_grid.py49492%64–65, 132, 135
   monte_carlo.py93397%79, 101, 230
   qmc.py20195%67
   rng.py60788%71–74, 93–94, 102
   simpson.py29390%52, 71–74
   utils.py1311192%35–36, 102, 173, 177, 184, 196, 198, 204, 299, 306
   vegas.py167498%106–107, 212, 359
   vegas_map.py103892%247–257
   vegas_result.py25196%22
torchquad/utils
   deployment_test.py1926367%53–55, 65–66, 69–70, 78–79, 91–93, 164, 175–176, 180–181, 183–184, 196–199, 211–213, 221, 232–234, 237, 247, 250, 253, 256–259, 267–275, 280–288, 297–306, 316–323
   enable_cuda.py15660%17–22
   set_log_level.py18478%56–57, 72–73
   set_precision.py34974%35–38, 50–51, 70–72
TOTAL129514189% 

Tests Skipped Failures Errors Time
205 0 💤 0 ❌ 0 🔥 2m 25s ⏱️

…ights

Review follow-up. Applying the weights out of place stopped the silent downcast
but introduced a silent upcast, which the fail-hard rule objects to just as much.
A dtype mismatch is now warned about, matching the backend-mismatch warning ten
lines above it. The check compares mantissa width, so a complex128 integrand
against float64 weights stays quiet -- it loses no precision -- while complex64
or float32 against float64 does not.

The no-mutation and dtype guarantees move from an inline comment into the
evaluate_integrand docstring: it is public API, and the tutorial shows users
calling it directly when writing custom integrators.

In gradient_test.py the three parallel lists become one list of tuples, so they
cannot drift out of sync again rather than being caught by an assert. The
exponential case now skips MonteCarlo and VEGAS, which never reach the weights
branch and were already covered by the four preceding integrands, and its bound
tightens from 5e-2 to 1e-4 -- the former was set by Monte Carlo and would have
passed a several-percent error in weight application on the deterministic rules.

The changelog entry drops the test-suite detail, which users do not ship, and
splits the dtype change into Changed where it belongs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YP9oDLmw636HhPSAD6Nv3p

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The precision table compares storage width rather than significand precision: bfloat16 and float16 are both mapped to 16 even though bfloat16 is less precise. A bfloat16 integrand with float16 weights therefore skips this warning. Please rank by actual mantissa precision and add that mixed-dtype case to the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants