[the-cashman]

Draft, proof of concept: not for merging as is. This is the code as it runs on our own Lichii-based branch, put on current nx2 so it can be read and compared. It goes with the coordination issue #899, which asks which translation cache nx2 wants and where it should live. Depending on the answer, the useful part would be redone small and on top of #781's precache (or on another cache format). This exact branch was not build-checked or re-measured on current nx2. The commits were moved from our branch with conflicts resolved by hand (mesh emulation from #569 in vk_graphics_pipeline/pipeline_helper, the cache carry-over and record lock in vk_pipeline_cache/shader_environment). The measurements below are from our branch.

What it does

Switch 2 titles stutter the first time a new shader is drawn. The emulator translates the stage on the GPU thread, then the driver compiles the pipeline. This removes both ahead of time, from the user's own copy of the game, with no recorded play:

• Per-stage translation cache (<shader dir>/<title id>/vulkan_ir.bin, nx2_ir_cache.*). A stage's translation depends only on its own program and its environment answers, not on the pipeline key. So the IR after TranslateProgram is stored per stage, together with every answer it used.
• Precompile (EDEN_NX2_PRECOMPILE=<pack>, RunPrecompile in vk_shaderwarm.cpp). During boot, on low-priority threads, every stage program of a pack is lowered and translated with default answers (nx2_precompile_env.*): 2D float textures, mapped storage pointers, the program's own bank-1 constants. Each stage is then checked: the stored form must give the same SPIR-V. The pack lists program blobs found by a generic scan of the game's files (outside the tree; see "Not included").
• Checked on every use (TryCachedTranslation). A record applies only if every recorded answer is the same live: storage pointers (direct and pointer-table), texture queries asked again through the live handle's constant-buffer words, texture bound buffer, viewport transform, local memory size. Otherwise the stage is translated as today.
• Verify switch: EDEN_NX2_IR_CACHE_CHECK=1 translates every hit live as well and compares the SPIR-V. It logs NX2PC check <hash> (vs|ps|cs|task|mesh): cached translation same as live or DIFFERS from live, and a difference uses the live result. EDEN_NX2_IR_MISMATCH_LOG=1 names the answer that rejected a record.
• Learning (EDEN_NX2_IR_LEARN=1): a stage translated live is appended with the same checks, so it costs one translation per build, not one per session.
• Storage-pointer variants: a record per mapped/null combination of a stage's pointers, up to 6 pointers (EDEN_NX2_PRECOMPILE_POINTER_VARIANTS, _POINTER_MAX).
• Lookups read only the records' checks and keep an LRU of decompressed programs.
• Driver side (EDEN_NX2_GPL=1, vk_gpl*, vk_scheduler_rendering.cpp): graphics pipeline libraries with per-stage descriptor sets and dynamic rendering. The precompile creates each stage's library once so the driver's disk cache holds it, and builds each compute pipeline with a known launch once. This is a separate GPL implementation from vulkan: pipeline libraries + shader warm-up from the game files (first-visit shader stutter) #579's and is here only because the measurements used it. nx2 should keep one (see Shader precompile: a per-stage translation cache filled from the game files, on top of #781's precache (which form do you want?) #899).

Everything is off unless an environment variable or an existing vulkan_ir.bin turns it on. Hooks in existing files are marked nx2 shaderwarm; docs/shader-precompile.md lists every switch.

Each user generates their own cache from their own game. Nothing derived from a game is part of this PR or meant to be shared. vulkan_ir.bin holds translated game code, and its tag (format, git revision, Info/Inst sizes, driver, GPU) makes it apply only to the build and driver that wrote it.

Measurements (from our branch, not from this one)

AMD Radeon RX 7900 XT 20 GB (RADV, Mesa 26.2.6), Ryzen 7 5700X3D, Linux (Fedora 45 base, kernel 7.2.15), local clang build. Ocarina of Time, scripted headless route; user settings with async shaders on.

Our branch: Lichii nx2 merged up to ad477ef1 (first-session run, 2026-11-08; the next-session run was one build earlier, 2026-11-07), plus this stack and our other, unrelated fixes. Each run started from a precompiled vulkan_ir.bin + Mesa cache and an empty vulkan.bin, with GPL and learning on:

	first session after the precompile (2026-11-08)	next session, learned records added (2026-11-07)
stage uses served from the cache	2,242	2,197
graphics stages translated live	16 without a record + 42 whose record didn't match (all learned)	1 (learned)
compute stages translated live	26	0
graphics pipeline builds on the GPU thread over 16.7 ms	0 (262 over 2 ms, 975 ms total)	0
crashes	0	0

• Verify mode, same branch: 82 mesh, 48 task, 550 vertex, 642 pixel and 69 compute cache uses compared with their live translation: 0 differed. Earlier, before texture queries were asked again through live handles, 33 of 1,496 pixel uses differed. After that change, 0 of 1,453.
• Precompile cost, 2026-11-08, 8 low-priority threads: 345.5 s for 46,877 stages. That produced 262,461 records with pointer variants (vulkan_ir.bin 6.9 GB), 225,468 stage libraries and 34,308 compute pipelines. The Mesa cache already held most libraries from earlier runs (0.13 ms each), so a cold run takes longer. Translation alone, without variants, took 14-16 s for 42,781 stages (older build); that file is ~0.55-0.65 GB.
• Older build (Eglantine nx2 0857e8b4 + this work at the time, before the Lichii port), 2026-11-06, async off, empty caches vs. precompiled, two runs each, interleaved: the driver-compile time frames waited on fell from 17.2 to 2.8 s, and frame waits from 5.0 to 1.2 s. GPU-thread build time halved. Gameplay frames over 100 ms went from 18-22 to 9-11 (the rest are not shader builds).
• Cold Mesa cache: the boot froze ~60 s at the game's first movie, on one compute pipeline. After the precompile there was no gap over 5 s (Precache shaders before game start (compute, from #579/#741) #781 now covers that compute case too).
• IR cache alone, without GPL: it halves GPU-thread build time but barely changes frame times with async off, because the driver compile dominates. With async on, frames over 50 ms went from 95 to 86.

Not included / limits

• The pack builder. The scan that makes the pack is a Python tool on our side. It reads every file of the game, undoes zstd with the game's own dictionaries, and finds NVN2 program headers by their layout and code endings; no per-game knowledge. Against the exact archive parser it found 12,681/12,681 programs in OoT, 3,249/3,249 in Mario Kart World and 2,321/2,321 in Fire Emblem. In-tree, this would be Precache shaders before game start (compute, from #579/#741) #781's archive walker, which already yields vertex, fragment and compute records, or a C++ port of the header scan for games whose containers the walker doesn't read.
• Not tested here: Windows, MSVC, NVIDIA, AMD's Windows driver; a build of this branch; an A/B against Precache shaders before game start (compute, from #579/#741) #781's precache.
• Cache invalidation: the file is tied to the git revision, so every build reruns the precompile. Upstream would want a translator-identity tag, as in daniel-lynch's cache.
• Pointer variants are large (10x the file) for a modest gain (live compute translations in a first session 29 → 21). They should stay off, with learning covering the rest.
• Not taken from our branch: the commit that makes precompiled stages follow shader_environment: read NVN's draw parameters from the draw (Ocarina of Time vegetation on AMD) #806's draw-parameter replacement, because shader_environment: read NVN's draw parameters from the draw (Ocarina of Time vegetation on AMD) #806 isn't in nx2.
• No unit tests.

Related work

#781 (@zortos293, from #579/#741 by @PantACRO4life), #895 (@zmckenney), and daniel-lynch's per-stage translated-shader cache on his fork. #899 compares them and asks which direction to take.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn