Skip to content

Assemble the far field on the device — the build is now faster than before WP2 - #68

Merged
TobiBu merged 1 commit into
perf/svgd-pallas-nearfieldfrom
perf/svgd-device-far-assembly
Sep 5, 2026
Merged

Assemble the far field on the device — the build is now faster than before WP2#68
TobiBu merged 1 commit into
perf/svgd-pallas-nearfieldfrom
perf/svgd-device-far-assembly

Conversation

@TobiBu

@TobiBu TobiBu commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Stacked on #65, which flagged this as its own main cost.

#65 made the update 34–40× faster and the build 21 % slower — 633.8 ms against 524.7, all of
it host-side — because the leaf-major partition does two orderings of 2.8 million entries where
the per-particle expansion did one np.repeat. Every input it needs is already a device array
when assemble_svgd_topology receives it, so none of that had to be on the host.

build at N = 10⁵, float64 device host total
per-particle expansion (before #65) 366.5 ms 158.3 ms 524.7 ms
leaf-major, host assembly (#65) 362.5 ms 271.3 ms 633.8 ms
leaf-major, device assembly 364.5 ms 69.6 ms 434.0 ms

The host half falls 3.9× and the whole build 1.46×, leaving it 90.7 ms faster than before
the leaf-major change rather than 109 ms slower. So "never materialise M" is a build-time win
after all — it just needed the assembly to follow the representation onto the device. The update
is untouched (φ 35.11 vs 34.83 ms, φ+∇ 121.60 vs 121.43): this is assembly only.

Two changes that made it simpler, not harder

  • The monopole table is keyed by node id, not compacted to the distinct far source nodes.
    That deletes the dense remap, and it makes the table's shape num_nodes — fixed by
    (n, leaf_size) — instead of a data-dependent F, so it no longer churns between rebuilds,
    for the same reason WP6: static capacities — 16x on a sampler that rebuilds every step #64's static capacities mattered. Roughly twice as long, three scalars a row.
  • Pairs the policy dropped get a leaf count of zero rather than a boolean filter, because a
    filter has a data-dependent length and would need its own synchronisation. They expand to
    nothing. Exactly one scalar now crosses to the host: the total entry count, which is a
    static shape everything downstream needs anyway.

Why the numpy path is still here

Below 16 384 accepted far pairs the dispatch and the synchronisation cost more than the work they
replace — the same shape of threshold the old expansion had. It is now semantically identical
to the device one, down to the node-id keying, so test_the_device_and_host_far_assemblies_agree
builds both and compares them rather than trusting each alone. Entry order within a leaf is free
(the entries are summed), so that comparison is per-leaf multiset.

test_the_transpose_csr_is_consistent_with_the_forward_one pins far_entry_perm directly — that
it is a permutation, that it groups the source list, and that it matches the node offsets. §6.3 of
the report records why: getting that map backwards leaves every value correct and the gradient
wrong by 88 %, and it slipped through once already.

One guard reads better as a result. With the table keyed by node id it is never empty, and a
capacity="pow2" padding entry names node 0, whose monopole exists and is masked by
far_leaf_live — so the guard is back on the entry list, and the empty-table trap it was working
around cannot recur in this form.

616 tests green. Evidence: reports/YGGDRAX_pallas_nearfield_report.md §6.5.

🤖 Generated with Claude Code

…faster than before WP2

The leaf-major far partition made the update 34-40x faster and the build 21 %
slower: 633.8 ms against 524.7, all of it host-side (271.3 against 158.3),
because it does two orderings of 2.8 million entries where the per-particle
expansion did one `np.repeat`. Every input it needs is already a device array
when `assemble_svgd_topology` receives it, so none of that had to be on the host.

    build at N = 1e5, float64, bench/svgd/scaling.py

      per-particle expansion        dev 366.5  host 158.3  total 524.7 ms
      leaf-major, host assembly     dev 362.5  host 271.3  total 633.8 ms
      leaf-major, device assembly   dev 364.5  host  69.6  total 434.0 ms

The host half falls 3.9x and the whole build 1.46x, which leaves it 90.7 ms
FASTER than before the leaf-major change rather than 109 ms slower. So "never
materialise M" is a build-time win after all -- it just needed the assembly to
follow the representation onto the device. The update is untouched: phi 35.11
against 34.83 ms, phi+grad 121.60 against 121.43.

Two changes made this simpler rather than harder, which is usually a sign the
first version was carrying something it did not need:

* **The monopole table is keyed by node id**, not compacted to the distinct far
  source nodes. That deletes the dense remap, and it makes the table's shape
  `num_nodes` -- fixed by (n, leaf_size) -- instead of a data-dependent F, so it
  no longer churns between rebuilds, for the same reason static capacities
  mattered. The table is about twice as long and every row is three scalars.
* **Pairs the policy dropped get a leaf count of zero** rather than a boolean
  filter, because a filter has a data-dependent length and would need its own
  synchronisation. They expand to nothing. Exactly one scalar now crosses to the
  host: the total entry count, which is a static shape everything downstream
  needs anyway.

The numpy path stays for partitions below 16,384 accepted far pairs, where the
dispatch and the synchronisation cost more than the work they replace -- the
same shape of threshold the old expansion had. It is now semantically identical
to the device one, down to the node-id keying, so a test builds both and compares
them rather than trusting each alone. Entry order within a leaf is free, since
the entries are summed, so that comparison is per-leaf multiset.

One guard reads better as a result. With the table keyed by node id it is never
empty, and a `capacity="pow2"` padding entry names node 0, whose monopole exists
and is masked by `far_leaf_live` -- so the guard is back on the entry list,
where it belongs, and the empty-table trap it was working around cannot recur in
this form. The test that pinned that trap now pins the new contract instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@TobiBu
TobiBu force-pushed the perf/svgd-device-far-assembly branch from a8b66b5 to 6196a50 Compare September 5, 2026 19:36
@TobiBu
TobiBu merged commit 7dc4feb into main Sep 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant