Move kernel compilation to build time

[mkeshavaNV]

Summary

slangpy generates and compiles a wrapper kernel module per call site at
runtime, in every process, with no first-class way to precompile that
work and ship it as a build artifact.

For any application with more than a handful of distinct call sites, each
new Python process pays seconds-to-minutes of slang→PTX codegen + CUDA
driver JIT before the first dispatch returns. Subsequent dispatches in
the same process are fast (in-memory pipeline_cache hit), but every new
process restarts cold.

slangtorch — the older C++-extension-based binding — solves the same
problem by moving all compilation to build time. There is no equivalent
flow in slangpy.

slangtorch's build pipeline:

   .slang source
       │
       ▼ slangc -target torch-binding   (build time)
       │
       ▼ generates a torch C++ extension source (.cpp / .cu)
       │
       ▼ nvcc + g++                     (build time)
       │
       ▼
   lib<name>_slang_cc.so   (built once, on disk, shipped as build artifact)

At runtime:

import importlib
m = importlib.import_module("path.to.libfoo_slang_cc")        # dlopen + symbol lookup
m.my_kernel(...).launchRaw(blockSize=..., gridSize=...)       # straight CUDA dispatch

First-call cost: a fraction of a millisecond — just dlopen and the
CUDA driver loading PTX into a module. The driver's PTX→SASS JIT is
itself cached in $CUDA_CACHE_PATH (default $HOME/.nv/ComputeCache),
so it's amortised across processes.

The Python slangtorch package additionally provides
slangtorch.loadModule(...), which runs slangc + nvcc + ninja once
and caches the resulting .so on disk; subsequent loadModule calls
reuse the existing build. Same model — compile once, persist, reload
fast.

What slangpy does today

slangpy's dispatch model:

   module.my_kernel(args)
       │
       ▼ slangpy Python layer synthesises a wrapper slang source per call site
       │
       ▼ session.load_module_from_source(hash, source)   (slang frontend)
       │
       ▼ session.link_program([wrapper, kernel_module])  (generic specialization)
       │
       ▼ device.create_compute_pipeline(program)         (slang → PTX)
       │
       ▼ cuModuleLoadData(ptx)                           (driver PTX → SASS JIT)
       │
       ▼ kernel dispatch

Every step from "wrapper source synthesis" through "PTX codegen" runs
on the first dispatch, in every process. None of it is currently
cacheable across processes through any first-class API:

• The Python wrapper synthesis runs every process (cheap, ~ms).
• The slang frontend (load_module_from_source) runs every process — the
  input is the freshly-generated wrapper string, which slang has to parse
  from scratch.
• link_program runs every process.
• device.create_compute_pipeline → slang → PTX runs every process
  unless Device.shader_cache_path is set and the path strings slang
  stores in IR are stable across processes — typically not the case under
  sandboxed tests, build systems with per-invocation paths, or any setup
  that doesn't pin slang sources at a fixed absolute path.
• cuModuleLoadData → driver PTX→SASS JIT runs every process unless
  CUDA_CACHE_PATH points at a writable directory.

The dominant cost for typical workloads is the slang→PTX codegen for
specialised generic templates. Indicative numbers from a workload with
~30 distinct call sites (mix of plain entry points and heavy generic
specialisations) on an A40:

call site type	per-wrapper first-call cost (cold cache)
simple kernel (no generics)	~3 s
medium generic	~5 s
heavy generic specialisation (e.g. Foo<3, 45, 5>)	25–75 s

Aggregate first-process startup cost in this workload: ~7 minutes
before the first iteration of training begins. The same workload with
the equivalent kernels built via slangtorch_library pays near-zero
startup cost.

Solution

A first-class build-time compilation flow analogous to
slangtorch_library. Any of the following shapes would be sufficient:

[jkiviluoto-nv]

@nv-slang-bot Mukund (mkeshavaNV) won't be returning to this work for a while. Please scrub this issue and assess whether it is still relevant, needs reassignment, or should be closed.

[nv-slang-bot]

Scrub & reassess (requested by @jkiviluoto-nv). Verified against main @ 507b4cf, confirmed identical to the live remote head. Verdict: keep open, unowned — this needs a roadmap decision from you, and I'm not going to manufacture a recommendation in place of one.

• Status & premise correction — "reassignment" is the wrong frame for this one. This issue has no assignee, no labels, and no milestone: fields read directly, assignees: []. @mkeshavaNV authored it (2026-05-26) but it was never assigned to him or anyone. So unlike Bool size mismatch when using CUDA or Metal device. #274/Add bool dtype support to slangpy native Tensor #899/able to wrap a 'raw entry point' for the backwards pass, and detect that if the forwards function is marked [CUDAKernel], so should the backwards one be #822 in this fan-out, nothing here became orphaned by his departure — it was unowned from the day it was filed, which is a different problem with a different remedy. It is also the only item in the fan-out that is not a bug or a checklist item: it's an architecture/performance proposal, so "still reproduces?" is the wrong test and "is this still the direction we want?" is the right one — and that question isn't answerable from the code.
• Q1 — Does the described cost still reproduce at HEAD? The mechanism is intact, unchanged, and I can confirm every structural claim in the body. The per-call-site runtime pipeline is exactly as described: the wrapper source is synthesised per call site, then session.load_module_from_source(hash, code) at calldata.py:509 parses a freshly-generated string every process, then link_program, then create_compute_pipeline. The cache that makes repeat dispatches fast is pipeline_cache, a plain in-process dict — declared self.pipeline_cache: dict[str, Pipeline] = {} at module.py:84, cleared at :197, read at calldata.py:495 and dispatchdata.py:176. It has no persistence layer, so every new process starts cold, precisely as the body states. What I did not do: reproduce the timing numbers. The 3 s / 5 s / 25–75 s per-wrapper figures and the ~7-minute aggregate are from a ~30-call-site workload on an A40; I have no equivalent workload and no A40, and building one is not a scrub-scale task. So I am confirming the mechanism, not the magnitudes — treat the table as the reporter's measurement, unverified by me.
• Q2 — Has anything landed since May that reduces this? Two things came close, neither addresses it. ① PR Add CacheWriter #1036 "Add CacheWriter" (merged 2026-06-30) moves persistent shader/module cache writes onto a background worker — it reduces write latency on the foreground compile path, not the cross-process cold-start cost. ② PR Honor deferred target compilation option #1013 "Honor deferred target compilation option" (merged 2026-06-03) exposes defer_target_compilation, used at calldata.py:513-514 and dispatchdata.py:202-203 — it defers target codegen rather than eliminating or persisting it. Device's shader_cache_path and module_cache_path still exist (device.cpp:363-364) with exactly the caveat the body already documents: they help only if slang's IR-embedded path strings are stable across processes. So the gap the issue describes is unchanged at HEAD. No PR proposes build-time wrapper compilation.
• Q3 — Existing coverage? One closely-related issue, already closed, and it does not subsume this. Support precompiled modules in slangpy #509 "Support precompiled modules in slangpy" (closed 2025-11-17) delivered .slang-module loading — verified in-tree at slangpy/tests/device/test_precompiled_modules.py, which compiles with slangc and loads a precompiled module without its source present. The distinction that keeps Move kernel compilation to build time #1001 alive: Support precompiled modules in slangpy #509 precompiles the user's own .slang modules; Move kernel compilation to build time #1001 is about the wrapper slangpy generates itself, per call site, at runtime — which by construction cannot be precompiled ahead of time today, because it doesn't exist until the call is made and is loaded from an in-memory string (calldata.py:509). That is the actual unsolved piece, and it is not a duplicate.
• Q4 — Should it be closed? I'm explicitly declining to recommend closure, and I want to be clear that's a judgement about evidence, not diplomacy. "The author left, nobody was ever assigned, therefore close" would be reasoning from process signals to a technical conclusion — and every process signal here is weak: the lack of assignee reflects that it was never triaged, not that it was considered and rejected. Meanwhile the technical substance is verified intact: the cold-start path is unchanged, no mitigation has landed, and the closed precompilation issue addresses a different surface. The honest position is: the problem is real and unaddressed, and whether it's worth solving is a roadmap call only you can make. The body's "Solution" section is also truncated mid-sentence — it ends at "Any of the following shapes would be sufficient:" with nothing following, so the proposed designs were never actually written down. That's worth knowing before anyone assesses feasibility.
• Next action — for you, not for me: ① decide whether build-time kernel compilation is on the roadmap. If yes → assign an owner, label, milestone, and ask the reporter (or whoever inherits the workload) to complete the truncated Solution section. If no → close as a roadmap decision with that rationale stated, so the concern is on record as considered-and-declined rather than quietly dropped. ② Either way, triage it into a real state — an untriaged, unlabelled, unmilestoned issue is invisible to every future sweep, which is how this sat 2½ months. Blocker: a human roadmap decision. No labels, assignee, milestone, or state were changed by me.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}

[nv-slang-bot]

Scrub follow-up — additive to the assessment above, which I re-verified against main @ 507b4cf (still the live head at time of writing). Its verdict stands: keep open, unowned, needs a roadmap decision. Three things it doesn't contain.

• Correction to my own earlier wording — the Solution section wasn't truncated, it was never written. I need to fix a causal claim I made loosely. The body ends mid-thought at "Any of the following shapes would be sufficient:", but it is 4,054 characters against GitHub's 65,536-character limit — about 6% of capacity, and userContentEdits.totalCount is 0 with lastEditedAt: null, so no stored revision ever held more text. Nothing was cut off by the platform. "Ends unexpectedly" is an observation; "was truncated" is a cause, and the evidence supports the first and refutes the second. The practical difference: this isn't lost data to recover, it's a design list that was never recorded.

• ⛔ The bigger finding: slangtorch_library — the reference design this whole issue is premised on — does not appear to exist under that name. The body says the solution should be "analogous to slangtorch_library" and that the same workload "built via slangtorch_library pays near-zero startup cost." I searched for it and found: 0 hits in GitHub-wide code search, 0 in shader-slang org code search, and exactly 1 issue-search hit — this issue's own body. Over the slangpy repository's full history (1,644 commits, all refs), git log --all -S'slangtorch_library' returns 0. Those zeros are backed by live positive controls, so they aren't dead queries: loadModule org:shader-slang → 654 hits, slangtorch org:shader-slang → 50, and in-repo -S'load_module_from_source' → 56 commits, -S'pipeline_cache' → 10. What slangtorch actually exposes is loadModule, clearPersistentShaderCache, clearSessionShaderCache, clearShaderCaches — no *_library entry point; slangc -target torch-binding is real, and building runs through PyTorch's cpp_extension (_write_ninja_file_and_build_library), while compileAndLoadModule is runtime JIT with a persistent on-disk cache, not a build-time rule. (A real slang_library CMake function exists, but in shader-slang/slang for stdlib modules — unrelated.) Fact: no public slangtorch_library. Hypothesis, clearly labelled: it's shorthand for that ninja/cpp_extension flow, or an internal NVIDIA-side wrapper over it. Either way, the baseline the ~7-minute-vs-near-zero comparison rests on cannot be verified from public code as written — whoever picks this up should establish what the comparison target actually was, and that question is answerable by people other than the original author.

• Still relevant, verified at HEAD rather than from the PR record. The cold-start mechanism is unchanged: the wrapper is synthesised and parsed from an in-memory string every process (calldata.py:509), and pipeline_cache is a plain in-process dict (module.py:84, cleared :197) with no persistence layer anywhere. Both persistent caches remain opt-in with no default (device.h:157, :162; consumed device.cpp:98-118). The body's path-instability critique also holds structurally: absolute include paths enter the session descriptor (shader.cpp:434-435), the digest is computed from it (:532), and that digest becomes the cache directory (:545) — with no path normalization in the tree — so a change in include-path strings misses the module cache entirely, exactly the sandboxed-test / per-invocation-build-dir case described. No build-time flow exists (slangpy_library, build_kernels, precompile_kernels, aot_compile → 0 files across CMake/pyproject/tools; the wheel ships .slang source). Nothing since has closed this: Add CacheWriter #1036 moves cache writes to a background worker (latency only), Honor deferred target compilation option #1013 defers target codegen rather than persisting it, Add persistent cache implementation based on LMDB cache #561 predates this issue by 7.5 months and is the very mechanism the body argues is insufficient, and Package Slang standard module in wheels #1034 ships Slang's prebuilt stdlib, not slangpy kernels. Unverified, and I want to be explicit: the 3 s / 5 s / 25–75 s and ~7-minute figures are the reporter's measurements on an A40 workload I don't have — I'm confirming the mechanism, not the magnitudes. Notably the in-tree slangpy/benchmarks/ppisp/ slangpy-vs-slangtorch comparison measures steady state with warmup, so cold-start cost is structurally unmeasured in CI.

• Scope note for @jkiviluoto-nv, since this sweep asked about reassignment on 34 issues. This one has never been assigned — zero assigned/unassigned events, ever — so it needs a first-time owner, not a reassignment. Across both repos the sweep covered 39 open items touching @mkeshavaNV (12 slangpy issues + draft PR [DNS] Dev/mkeshava/benchmark test #904, empty body, untouched since 2026-05-18; 23 slang issues + 3 draft PRs stale 80–161 days). Of those, 27 are still assigned to him and 5 are genuinely unowned (slangpy Move kernel compilation to build time #1001, DiffTensorView support in slangpy #510; slang #9004, #8527, #7209) — while able to directly call an entry point without wrapping it in a trampoline with a CallData structure if it is already tagged as a compute shader or cuda kernel #820/able to generate a minimal wrapper for an entry point when the only thing missing is the compute shader / cuda kernel tag #821 already moved to @ccummingsNV back in March, so the sweep's premise doesn't hold uniformly. One gap the sweep missed: Modules: slangc -dump-module failing for some slang-modules slang#6664 is assigned to him and received no scrub request. Finding an owner for this issue alone leaves the rest dark.

• Next action — human, and not blocked on the departing author. ① Decide whether build-time kernel compilation is on the roadmap. If yes, assign an owner and settle the slangtorch_library baseline question first — the tractable subsets are visible from the code (persist the wrapper+pipeline across processes; stabilise the session digest against include-path variation), and only a true build-time rule needs a call-site declaration mechanism that was never designed. If no, close it as a stated roadmap decision so the concern is on record as considered rather than quietly dropped. ② If @mkeshavaNV is still reachable, the highest-value ask is what slangtorch_library referred to, not "finish the Solution section" — though note his last authored activity here was 2026-04-07 (commit) and 2026-03-30 (comment on an issue he owns), so I'd treat a reply as unlikely and not gate on it. Also worth knowing: slang#9004 (his, open, unowned) reports UseUpToDateBinaryModule breaking precompiled-module loading, which any build-time flow would hit. Blocker: a human roadmap decision. I changed no labels, assignees, milestone, or state, and edited nobody's comments.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}