Repository navigation
vulkan: pipeline libraries + shader warm-up from the game files (first-visit shader stutter) - #411
PantACRO4life wants to merge 7 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…pute build logs With EDEN_NX2_FAST_FIRST=1 a pipeline built while the game runs is first created with VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT and used as soon as the driver returns it. The optimized build follows on a worker and replaces it at a later draw or dispatch. On a demanding Switch 2 test title the first-visit stutter is the driver compile (draws held up to 7 s of every 10 s with an empty driver cache); this trades it for a slower shader during that time. Compute pipelines built synchronously log their driver build time when it is over 100 ms (Nx2Perf compute_build), since a dispatch waits for the whole build. Off by default; nothing changes without the variable.
EDEN_NX2_HEAVY_SKIP=<N> skips the draws of a native pipeline with at
least N converted instructions over all stages while the driver builds
it, instead of holding every draw behind it ("1" picks 4000). On a
demanding Switch 2 test title every driver build over 500 ms was above
4000 instructions, and those made about 90% of the compile time.
A heavy draw is still held when skipping it could lose data for good:
the pipeline writes storage or uses transform feedback, or its render
target was not cleared this frame, or is a new small target (a baked
texture rather than a scene-sized image redrawn every frame). Clears
are reported to the pipeline cache for this (NoteClear). Held draws
build their own pipeline instead of queueing behind skipped builds.
Environments report their converted instruction count (CodeWords) so
the pipeline knows its size. Counters are logged every 10 s, and held
draws over 100 ms log the pipeline size, the reason they were held and
the render target size (Nx2Perf held_draw).
Off by default; nothing changes without the variable.
EDEN_NX2_COMPUTE_WARM=<file> loads compute programs prepared offline from the game's own shader archives (pipeline cache format, no recorded reads) and builds them on 4 low priority threads after the disk cache has loaded. The pipelines are thrown away; they only fill the driver's shader cache, so a dispatch that later converts to the same SPIR-V finds the driver build ready instead of waiting seconds for it. Programs already in the disk cache are left out. Reads a file record does not hold are answered with guesses (FileEnvironment::SetGuessMissing / VideoCommon::GuessMissing) instead of an error. With the defaults about 95% of the programs converted to the same SPIR-V as the runtime records on the test title. Tools: - shader-sweep --guess-missing [--guess-cbuf0, --guess-cbuf-other, --guess-texture-type, --guess-pixel-format, --guess-storage-pointer, --guess-storage-pointer-other] translates records with missing reads the same way, and accepts cache version 53. - shader-sweep --override-out FILE writes the lowering as a program override record and reports the program's native_key. - tools/nx2/make_compute_warm.py builds the warm-up file from a static shader dump (extract_static_shaders.py output). Off by default; nothing changes without the variable.
…GPL) With EDEN_NX2_GPL=1 and a device that supports VK_EXT_graphics_pipeline_library with fast linking, graphics pipelines are linked from per-stage libraries instead of being built whole. Without the variable the device is created exactly as before (the extension stays out of the extension tables). Each stage gets its own descriptor set (set index = stage index, independent sets, plain descriptor sets, Bindings::descriptor_set in the SPIR-V backend) with bindings from 0, and the fragment stage's rescaling bits start at fixed indices, so a stage's SPIR-V and library do not depend on the partner stage's resources. Pre-rasterization and fragment shader libraries are cached in PipelineCache, keyed by the emitted SPIR-V hash, the stage's set layout, the state the part bakes in, the dynamic state list and the render pass; a worker that needs a library another worker is building waits for it. Vertex input and fragment output libraries are built per pipeline, then fast linked. EDEN_NX2_GPL_OPTIMIZE=1 adds an optimized link on a worker, swapped in like the fast first build. A failed library or link falls back to a monolithic pipeline with the same sets. Task/mesh, transform feedback, static rasterizer discard and stages with too many textures or images for the split rescaling bits stay on the monolithic route. PipelineCache::WarmGraphicsLibraries builds a pipeline's shader libraries with no draw; with a single stage and no partner it builds only that stage's library. Nx2Perf gpl stats are logged every 10 s. Based in part on suyu-main 3157ae0ce0 and f80bbc701a (library cache built outside the lock, fast then optimized link, link statistics); unlike there, shader libraries are shared across partner stages.
EDEN_NX2_GRAPHICS_WARM=<file> (with graphics pipeline libraries on) builds the game's vertex and fragment programs as single-stage libraries in the background after the compute warm-up. The file is streamed (it can be over a gigabyte); worker threads take the next entry under the reader's lock. Libraries built by the warm-up are not kept (GraphicsPipelineLibraryCache::SetNoRetainOnThisThread), so only the driver's disk cache keeps them and a later pipeline with those stages costs a link. tools/nx2/make_graphics_warm.py writes the file from a static shader dump: one single-stage entry per unique vertex or fragment program, with draw state taken from the most common state in pipeline caches the game recorded (--runtime-cache). --validate compares the generated entries' SPIR-V with the runtime records. Off by default; nothing changes without the variable.
…tus bar progress The warm-up files no longer need the offline scripts. A game list entry "Prepare shaders (warm-up)" (with Rebuild and Delete) runs ShaderWarm::BuildWarmFiles in a progress window with Cancel; games do not start while it runs. The builder: - reads the game's RomFS through the VFS (.nup base and update packages, other formats through the loader); - extracts shader programs with C++ readers for the archive formats (zstd with the game's dictionaries, DCX, LZ4, SARC, BND4, BFSHA/BNSH, effect bundles, CGSB, SHN, BAHS v11), a port of the PR zortos293#214 extractor with the same checks; - lowers and converts them in process with the runtime code layout, on low priority threads in program order, and writes the per-title warm-compute.bin and warm-graphics.bin atomically, with the same records make_compute_warm.py and make_graphics_warm.py write. Draw state comes from the title's v53 pipeline caches when there are any, else from built-in defaults measured on one test title. At boot the warm-ups load the title's warm files when EDEN_NX2_COMPUTE_WARM / EDEN_NX2_GRAPHICS_WARM are unset. A warm-up that ran to the end writes a .done marker (file size and write time, entries, driver version, device ID, cache version); a matching marker skips that warm-up at the next boot. Rebuild and Delete remove the warm files, markers and temporary files. Progress goes through ShaderNotify (SetWarmProgress / GetWarmProgress, lock-free, 4 updates a second) to the status bar, next to the shader build count: compute and graphics counts, with a percent when the total is known. A "Shader libraries" setting (use_graphics_pipeline_library) in the advanced graphics tab turns on graphics pipeline libraries like EDEN_NX2_GPL=1. yuzu_cmd gets a hidden --build-warm <game> <title id> <output dir> [--defaults | caches] that runs the builder without booting the game (test hook).
|
Sharing what we have measured on the same problem, in case it saves you time. We have been working the first-visit stutter on our fork (RTX 3080 Ti, Linux, driver 615.78, Ocarina) and landed on the same conclusion: the freeze is the driver compile, and the disable-optimization bit changes nothing. No wish to duplicate your work; take whatever is useful and ignore the rest. Compute compiles. Instrumenting the first dispatch, a new compute pipeline makes the dispatch wait 2 to 13 s; on one cold walk 26 waits added up to 49.7 s before the HUD, CPU time equal to wall time. The reason the compiles are slow is our SPIR-V: the global-memory helper functions (the address-against-every-storage-buffer tests, 300 to 870 instructions each) are called from dozens of sites and the NVIDIA compiler inlines every call. Marking helpers of 128+ instructions with 2+ call sites DontInline cut the compile of the 92 affected shaders from 246 s to 72.5 s (the 30k-word class from about 5 s to 0.27 s) and halved first-dispatch waits on a cold Field, Market, Deku Tree route (51 s to 22 s, waits of 1 s or more from 18 to 7). Running spirv-opt -O first made the compile slower. Caveat: a runtime cost of up to about 5% is possible and not settled yet. Branch: https://github.com/daniel-lynch/Lichii/tree/fix/compute-pipeline-stall (switch Warm-up from the game files. We do this today for the translation half: an index built from the game's shader archives maps programs to containers and scenes, and an offline producer translates them ahead of time into a product the runtime loads (-41% cold-session frame-blocking on our route; a pipeline cache recorded from play and a scene tour gives 0 runtime pipeline builds on a recorded route). Docs: Happy to run anything on NVIDIA Linux or share the recorded data if it helps. Measurements gathered with Claude Code's help. |
I am already about to push my PR so no worries. I have the code ready im just making sure its building correctly before pushing : https://github.com/PantACRO4life/Lichii/tree/perf/gpl-shader-warmup-wip |
1e57f0c to
239030d
Compare
…nditional, library params default argument)
|
Closing this draft: it has no code diff against nx2, only a design write-up. Feel free to open a new PR when there is code to review. Thanks! |
|
|
…3#411, ff35f64) Off by default as in the pull request: EDEN_NX2_GPL or the Shader libraries setting, warm-up files from "Prepare shaders (warm-up)" in the game list, EDEN_NX2_HEAVY_SKIP, EDEN_NX2_FAST_FIRST. Heavy skip is one more reason to discard a draw under the native_async_draws modes; warm files use this tree's cache version and read the -frizti pipeline cache.
GPU timestamps take upstream's AmpereAnchor (zortos293#664) in place of the earlier present_ns form. zortos293#583's cooperative matrix flags go into MakeComputePipeline. zortos293#569's mesh emulation is combined with zortos293#411's pipeline libraries the way PantACRO4life's zortos293#579 does: a pipeline that emulates mesh shaders is built monolithically, and a warm-up build of one is skipped.
Draft, work in progress. Opening early to share the measurements and the direction, so nobody duplicates the work and reviewers can comment on the design before it lands. Not ready to merge yet.
Problem
On a first visit (no shader caches at all), a demanding Switch 2 test title freezes for 0.5 to 5.6 s each time it meets a new pipeline. Measured on an RTX 4090 (driver 617.14), i9-13900K, Windows, empty emulator cache and empty NVIDIA cache:
What did not help: VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT (NVIDIA ignores it: the "fast" build takes the full time, the optimized one after it 0.1 ms).
What NVIDIA caches (harness measurement)
Small standalone Vulkan harness, same SPIR-V as the emulator:
So a stage compiled once as a library serves every pipeline that uses it.
Can the stages be prepared before the game needs them?
Yes, from the user's own game files:
What this PR does (so far)
EDEN_NX2_GPL=1): pre-rasterization and fragment libraries cached and shared across pipelines, per-stage descriptor sets withVK_PIPELINE_LAYOUT_CREATE_INDEPENDENT_SETS_BIT_EXT, fast link at first use, optional optimized link in the background (EDEN_NX2_GPL_OPTIMIZE=1), monolithic fallback for unsupported paths (mesh/task, transform feedback, very large descriptor counts). First in-game run: fast links 0.008 to 0.011 ms on average, 0 library failures, 0 fallbacks. Structure based on suyu-main's GPL work (3157ae0ce0, 28c74ded03, f80bbc701a, c04197d8b0), extended so libraries are independent of the partner stage.EDEN_NX2_COMPUTE_WARM): the title's 4,794 compute programs are converted in the background after boot and built once so the driver cache holds them; the results are thrown away and never used for drawing, so a wrong guess only costs a normal compile. In game, after the warm-up: no compute build over 100 ms during a walk through new areas (before: dozens of 1 to 3 s builds). First warm-up 9 to 10 min on 4 low priority threads.EDEN_NX2_GRAPHICS_WARM): about 60,000 vertex and fragment programs built as single-stage libraries, released after each build so only the driver's disk cache keeps them (RAM stays flat).EDEN_NX2_HEAVY_SKIP): until a stage is warm, draws of pipelines over a size threshold are skipped while they build, with guards so draws that write storage, use transform feedback or draw into a texture that is not rebuilt every frame are still held. Second test area, empty caches: 32.6 -> 46.5 fps, p99 297 -> 42 ms, worst frame 2.5 -> 1.6 s (brief pop-in instead of freezes).Nx2Perf gpl,held_draw(size, reason, render target),compute_build, warm-up progress.First in-game impression (libraries + warm-up, empty caches)
Played while the graphics warm-up was still at about 40%: no visual issues seen, and the small props that used to pop in on first visit (rocks, grass, pots) were drawn right away. Walking from an open area into a dense one, which used to freeze repeatedly, had almost no hitches. Frame time numbers after the full warm-up will follow.
Still to do
Nothing from any title is shipped or shared: the warm-up files are generated on each user's machine from their own copy.
Related
#214 (playthrough hints, its archive extractor is the base of the shader dump), #224 (async discard).