Repository navigation
AMD FSR3.1 - FSR 4.1 - DLSS5 NR - #1335
notquiteog wants to merge 146 commits into
Conversation
Several open requests ask for DLSS, FSR and a proper upscaling technique (NotRexed#744, NotRexed#691, NotRexed#687, NotRexed#672), and two report the current frame generation (NotRexed#755, NotRexed#639). NotRexed#577 is adding FSR 4 Native AA on the finished image. Before more of it is written, this writes down where a temporal upscaler belongs in this renderer and in what order the pieces can land. The finding that decides it: OoT evaluates its own neural TAAU network (program fd029a317a9a4b94), which `taau_stand_in` in gpu/vk/vk_draw.cpp already recognizes and substitutes, because the translated network loses the blue channel. The game therefore already renders the 3D at a reduced internal resolution and resolves it temporally. A vendor upscaler takes that dispatch's place, with the existing stand-in as its fallback; the composited window image, where `upscale` and the frame interpolator work, has no depth, no motion and a HUD in its history. The document states the input contract such a backend must check before it runs, keeps the replay rules of design.md section 4, sets out six single-topic pull requests starting with a trace switch rather than a backend, and records what each vendor runtime permits: FidelityFX sources are MIT and need no user runtime, the Streamline headers are MIT but its libraries may never be committed or bundled, and Linux has no Streamline build so it needs NGX directly. It also says plainly that Ray Reconstruction is not implementable here: it needs albedo, normal, roughness and specular-hit-distance guides, and a search of gpu/ finds no such target in a rasterized game's replayed command stream. Docs only; no code or behaviour changes. `python -I -B tests/run_all.py` unaffected. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The log that reports a game network once per key (vk_draw.cpp) listed its sampled textures as extents and format numbers, leaving the reader to work out by hand which one a temporal upscaler would take for colour, motion and depth. Before an upscaler can be given that dispatch (docs/upscaling.md), the contract has to be read off the real thing, so this makes the log say it. gpu/vk/upscale_contract.h decides which sampled binding is which, and gpu/vk/upscale_format.h decides what a Vulkan format counts as. Motion and depth fix the render extent, since they are the two a network cannot sample at another size, and they must agree or nothing is reported: two that disagree mean the guess about which binding is which is wrong. The colour at that extent is the input. A zero extent is a binding the renderer could not resolve and is skipped rather than taken for motion. A colour at a larger extent is reported as "full-size" and nothing more. It is the network's own history, its output, or both, and draw_sampled cannot tell them apart, because it holds what a dispatch's descriptors sample *or* store in one list. Calling it the history would have been a guess printed as a fact; the seam that replaces the dispatch resolves the output image itself and can then say which it is. The contract header has no Vulkan include and reads nothing from guest memory; the format mapping is a header of its own rather than a static in vk_draw.cpp so that tests/upscale_contract_test.cpp covers it, which is the half most likely to be wrong: a format wrongly called motion is handed to an upscaler as motion. Also corrects this file's own claim from the previous commit: the compute stand-in replaces the network only where the device has no hardware f16 matrix multiply (or with RX_VK_NO_F16), and otherwise the network runs as itself. A replacement therefore cannot live inside taau_stand_in alone. No switch and no behaviour change: the extra work is one classification per network key, on a path that already builds a line once per key. Tested with python -I -B tests/run_all.py (88 passed; the 8 failures and 12 skips are the same on upstream/main here, all for a missing clang++ or SPIR-V headers), and each refusal in the classifier was confirmed to fail the test when removed. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs/upscaling.md row 2. A recognized upscale dispatch is now resolved once into an upscale::Request -- what the sampled bindings provide, the image it writes, and which binding the renderer hands over as colour -- and upscale::choose() says which backend takes it. A backend does not re-derive the dispatch for itself, which is what keeps FSR and DLSS from each growing their own copy of this. No behaviour change. The compute stand-in is the only backend, and it is a spatial resolve that needs none of the input contract, so it still runs wherever RX_VK_TAAU_KEYS asks for it, including where the contract is not met. choose() returns StandIn exactly when push descriptors are available and the key list asks for it, which is the same condition the first guard tested before, and tests/upscale_seam_test.cpp pins that: making the stand-in require the contract fails it. upscale::temporal_possible() is the gate a temporal backend has to pass. Complete inputs, an output that is resolved, writable and no smaller than the render extent, and the colour the renderer hands over being the colour the contract identified. That last clause does not hold yet, and the seam refuses rather than assuming it: the renderer passes dw.imgs[0], the first texture descriptor, while the contract indexes the textures the descriptor build resolved (draw_sampled). Those are different lists -- dw.imgs carries every texture descriptor, draw_sampled only the resolved ones -- so binding 0 of one is not binding 0 of the other. Establishing which descriptor is the contract's colour is the first job of the backend that needs it. The seam logs the refusal once per program with the output and input extents it resolved, so the next step reads what it needs from a run rather than guessing. One place now builds the binding list (upscale_bindings), so the per-network log and the seam cannot disagree about what a dispatch sampled. Tested with python -I -B tests/run_all.py: 89 passed, and the 8 failures and 12 skips are the same on upstream/main here, all a missing clang++ or SPIR-V headers. Each refusal in the gate was confirmed to fail the test when removed. The renderer itself is not compiled here: a clone carries no CPM dependency set, so the call site is reviewed against the guard it replaces rather than built. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…mpled The seam could not hand a temporal backend its colour input, because the list the contract was reading is not the list a backend is handed. draw_sampled records a sampled texture only where the GPU has written it (vk_draw.cpp, the gpu_written guard) and records every storage image as well, so its index i is not dw.imgs[i]. A backend wired to that correspondence would upscale whichever texture happened to be bound first, and the ghosting would be reported as a renderer bug rather than as a wrong binding. upscale_bindings now resolves the dispatch's texture descriptors itself, the way build_descriptors resolves them -- same view type, same ViewUse::Sample, same handle_is_storage_pointer guard for the case Fritchii's d603d98bb fixed -- so index i is dw.imgs[i]. The seam sets colour_binding from the contract, and temporal_possible() can now be satisfied rather than refusing by construction. The per-network log reports the same list, so its roles are descriptor slots too. Its entries are now "slot:WxH=role" rather than the extent-and-format of whichever textures draw_sampled happened to hold, which is what makes the log usable for deciding what a backend reads. No behaviour change: nothing new is recorded into a command buffer, and the stand-in still takes the same dw.imgs[0] and dw.imgs[ntex] it always did. The resolve is a view lookup for handles the descriptor build has just resolved for this same dispatch. Tested with python -I -B tests/run_all.py: 89 passed, with the same 8 failures and 12 skips as upstream/main here (a missing clang++ or SPIR-V headers). The renderer is not compiled here -- a clone carries no CPM dependency set -- so this is reviewed against the descriptor build it mirrors, and the formats and extents it reports want checking against a real run before a backend trusts them. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first half of an FSR backend (docs/upscaling.md row 3): its shaders. Adds AMD's FidelityFX SDK v1.1.4 GPU sources under third_party/fidelityfx-sdk, unmodified and MIT, with a provenance note, and scripts/fsr3_shaders.py to compile the eleven variants into gpu/vk/shaders/fsr3_*_comp_spv.h. Eleven, not eight: the eight passes, plus wave-free variants of both pyramids for devices without the subgroup operations the single-pass downsampler prefers, and an accumulate compiled with sharpening on. The option set lives in one place in the script because each option is a promise about the inputs -- HDR colour, low-resolution motion that is not jittered, reversed depth -- and the host code has to hand the passes what they were compiled for. This is the SDK-source route rather than AMD's FidelityFX API, so no runtime library is needed, none is included, and one code path serves Windows and Linux. NotRexed#577 vendors AMD's API headers for the other route under third_party/fidelityfx; if both land, one should go, and the doc says so. The headers are checked in, like every other shader in gpu/vk/shaders, so a clone builds with no glslang. tests/test_fsr3_shaders.py recompiles all eleven and compares, and validates each module with spirv-val, so the headers cannot drift from their sources and an invalid module is caught here rather than on a device. It is wired into run_all's tool table, and glslang-tools is added to the Linux CI step that had spirv-tools only. That costs about 1.3 MB of generated hex. Generating at build time would remove it at the price of a glslang dependency for every build; the doc records the trade rather than deciding it. No renderer change: nothing includes these headers yet. The host side that records the passes is the next commit. Checked here: all eleven compile and pass spirv-val, the check passes, and it fails as it should on a changed word in a header, a changed compile option and a missing header. python -I -B tests/run_all.py: 90 passed, with the same 8 failures and 12 skips as upstream/main (a missing clang++ or SPIR-V headers). Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Whether a denoising upscaler can ever work here -- NVIDIA's Ray Reconstruction or AMD's Ray Regeneration -- turns on one question: does this game write a G-buffer. Neither takes colour, motion and depth; both want a ray-traced renderer's guides, which means albedo, normals, roughness and specular hit distance. I have been answering "no" from one piece of evidence, that nothing in gpu/ names such a target, which is weak: the replay does not name what it replays. A deferred renderer writes its guides as several colour attachments of one extent in one pass, so that is what to look for. RX_VK_MRT_CENSUS=1 logs each distinct render target set once -- the colour attachment count, their formats and the extent -- from targets(), which resolves once per NOP_SET_RT rather than per draw, so the cost is one line per distinct set and nothing at all while off. Capped at 512 distinct sets so a pathological run cannot fill a log. Never more than one colour attachment means there is no G-buffer and the feature is settled as not applicable to this title. Three or four at one extent means guides may exist and identifying them becomes ordinary work. Either way the docs can then state a finding instead of an assumption. docs/upscaling.md also records what is actually deliverable per vendor, and splits FSR 3 and FSR 4 into two backends rather than two versions: FSR 4 is not source-available, ships as prebuilt signed libraries in AMD's SDK releases, and is reached by loading AMD's runtime and asking for a provider, on AMD hardware. The API it is reached through is the FSR 3.1 API, so row 3 is the path to it. No behaviour change while the switch is unset. Tested with python -I -B tests/run_all.py: 90 passed, with the same 8 failures and 12 skips as upstream/main here. The census needs a run to produce its answer, which needs a built game. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…le it A 240 s run of the opening with RX_VK_MRT_CENSUS=1: 136 distinct target sets, 491 presents. The test I set was "if the game never binds more than one colour attachment, there is no G-buffer" -- and it binds more. Seven sets bind six, two bind three, six bind two. Reading them settles it the other way. The six-attachment sets are 1x1 through 64x64 with every attachment B10G11R11: six cubemap faces down a mip chain, an environment or irradiance probe. The two- and three-attachment sets are 170x96 to 768x432 in single-channel float pairs or HDR pairs, which is post-processing at a quarter resolution or less. The main scene is one HDR target at 2048x1152, and the present is 2560x1440 RGBA8 sRGB. A deferred G-buffer is three or four attachments at the main render resolution carrying albedo, packed normals and roughness. Nothing here is that, so there are still no guides for a denoising upscaler and Ray Reconstruction and Ray Regeneration stay inapplicable here. The conclusion is the one I had been asserting; the argument I was asserting it from (no target in gpu/ is named "normal") was not a sound one, and the census is. Also confirms the rest of the series runs: the per-network log prints its roles and verdict for real networks (the movie upscalers report "no upscaler inputs", correctly -- they sample one 1920x1080 texture and nothing else), and the census costs one line per distinct set. The gameplay TAAU dispatch was not reached in this run, so the seam has not fired yet: with RX_VK_PRECOMPILE=0 the opening runs at about 2 fps because every pipeline compiles on demand, which is not an fps measurement and does not get past the titles. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first half of the backend: what an evaluation needs before one can be recorded. gpu/vk/vk_fsr3.cpp builds the eleven pass pipelines from the SPIR-V the previous commit checked in, the two samplers every pass declares at bindings 1000 and 1001, and the SDK's eighteen internal images. The resource table and the per-pass binding lists are written out in the SDK's order rather than derived, because they are the SDK's and not ours: ffx_fsr3upscaler.cpp fixes the formats and sizes, and each pass file numbers its own bindings. One wrong entry is a validation error at best and a wrong picture at worst. The one place that knowingly departs from the SDK is the luma history, allocated R8G8B8A8_UNORM because that is what the pass files declare as its storage format, where the SDK allocates RGBA16F -- a storage format qualifier has to match. supported() asks once whether the device has push descriptors and every storage format the images need, so a device that cannot run the passes says so before an allocation fails mid-frame rather than after. prepare() re-makes the images when the render or upscale size changes and tears everything down rather than leaving a half-built set behind. Nothing calls this yet, so there is no behaviour change. The dispatch half needs one input it cannot get from the renderer alone: device_to_view_depth and tan_half_fov come from the game's near plane and field of view, which for this title means reading its camera out of the dispatch's own constant buffer. That is the next commit, and the seam logs what it would need. Builds clean with RX_VK_ONLY=1, 0 warnings. Linux, RTX 5060 Ti, driver 615.71.09. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The other half: the constants, the clears, the barriers and the nine dispatches, following the SDK's ffx_fsr3upscaler.cpp rather than inventing an order. cbFSR3Upscaler is laid out field for field with a static_assert on its 148 bytes, because every pass reads it by offset and a field out of place is a wrong picture rather than a compile error. What is faithful to the SDK and worth checking in review: the jitter phase count moves one step a frame toward 8*(ratio^2) instead of jumping; the SPD counter, its mips and the previous-nearest-depth scratch are cleared every frame while the accumulation and frame info are cleared only on a reset; the Lanczos weights are built once and copied in; the two ping-pong sets swap on parity at the end; and RCAS only runs when sharpening is asked for, otherwise the sharpening accumulate variant writes the output directly. Three things specific to this renderer rather than the reference. Barriers are synchronization2, which is what this tree loads (vkCmdPipelineBarrier is not in vk_loader.h). The constant blocks live in one host-visible mapped buffer with a set of slots per frame in flight, so writing one frame cannot disturb a frame still being read. And descriptors are pushed per pass, with the two samplers at the bindings the SDK's GLSL fixes them to. upscale() refuses rather than guesses. No colour, depth or motion is a refusal, and so is a missing near plane or field of view: device_to_view_depth and tan_half_fov are built from them, and a guessed camera makes disocclusion wrong in a way that reads as a renderer bug. Nothing calls this yet, so there is no behaviour change; wiring it to the seam needs the camera read out of the dispatch's own constant buffer, which is the next commit. Untested beyond compiling: builds clean with RX_VK_ONLY=1, 0 warnings, Linux, RTX 5060 Ti, driver 615.71.09. No pass has executed, so nothing here is claimed to produce a correct image yet. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…scaler
A warmed run reached gameplay and the per-network log answered the question
this series was built on:
vk: network fd029a317a9a4b94: dispatch 32x36x1, 1 textures 1 images:
0:1536x864; no upscaler inputs
One sampled texture, one written image, nothing else. No motion, no depth. All
eleven networks in the game are that shape: at most two sampled textures, and
never a motion or depth target among them.
So OoT's upscaler is a single-image neural upscaler, not a motion-vector
temporal one, and docs/upscaling.md's premise -- that a vendor upscaler takes
that dispatch's place -- was wrong. FSR 3.1 and DLSS Super Resolution are
temporal. They need motion and depth, and this dispatch is never handed either,
so temporal_possible() refuses at it and will keep refusing however a backend
is wired. The doc now says so above the section it contradicts.
The gate doing exactly that is the one good outcome: it refused rather than
handing FSR whichever texture happened to be bound, which is the failure the
descriptor-order commit was about.
What the run does confirm is that the opportunity is real. The input is
1536x864 against a 2048x1152 scene, so the game already renders the 3D at 75%
and upscales it. The inputs for a temporal upscaler have to be assembled
instead of taken: depth exists on the scene passes and has to be carried to the
resolve, and motion exists nowhere as a target and has to be derived from the
camera, which ghosts on anything moving by itself. That is NotRexed#577's integration
point, and this is the argument for it.
Also in here, since they were written before the run: FSR is wired to the seam
behind RX_VK_FSR=1 (it cannot fire, by the above, but the path is complete and
compiles), and the camera is no longer a precondition. The reference
implementation documents that device_to_view_depth only scales FSR's
disocclusion thresholds and uses a typical near plane and field of view, so
refusing without the real camera was stricter than the evidence warrants.
Builds clean with RX_VK_ONLY=1, 0 warnings.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit left the camera as a typical value, which is what the
reference implementation settles for because DLSS hands it none. It does not
have to be a guess here: sead's perspective projection carries it, and its
matrix update is already a patch site (nx2/oot_wide.py, main+0x3485cc0, the
whole function f_834a5cc0).
Reading that function's lifted C gives the layout, each offset from what the
code does with it:
+0x98, +0x9C the near and far planes, loaded as a pair and swapped by the
byte at +0x0A before their difference is taken, so that byte is
a reversed-depth flag
+0xAC tan(fovy / 2): 1.0f / it becomes m11, which is 1 / tan(fovy/2),
so the field of view is 2 * atan(it)
+0xB0 the aspect ratio, divided into m00 (and where the ultrawide
patch stores its own)
+0xB4, +0xB8 the projection's offsets, each doubled into m02 and m12
x0 is the projection object throughout that function and x1 is the 4x4 it
writes, so one injected block there can publish the near plane, the far plane
and the field of view to host globals the renderer reads, the way
hos_wide_ratio already travels the other way. It is also the same data
camera-derived motion vectors need, which is what the integration point this
series is moving to requires.
Docs only.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the stand-in Two fixes that the first run found. **The camera is no longer a typical value.** nx2/oot_camera.py appends a block to sead's perspective projection update (main+0x3485cc0, f_834a5cc0), where x0 is the projection object, that publishes the near plane, the far plane and tan(fovy/2) to host/hos_oot_camera.c. The renderer reads them and gives FSR the real camera; without them the Inputs defaults stand, which only scales FSR's disocclusion thresholds. docs/upscaling.md has how each offset was identified from what the function does with it. The site is shared with oot_wide.py's camera-ratio patch, so this appends to lifter.after rather than replacing it, and lift.py installs the ratio first. Which of +0x98 and +0x9C is the near plane depends on the byte at +0x0A that the function uses to swap them, so rather than decide what that flag means this takes the smaller as the near plane and the larger as the far, and publishes nothing unless both are positive and ordered. **A backend that is not the stand-in has to be reachable without it.** My previous commit put the FSR path inside taau_stand_in, after a guard that returns unless taau_key matches -- and on a card with hardware f16 matrix multiply taau_key defaults to "-", because the real network runs there. So FSR was unreachable on exactly the hardware that can run it. That is the same flaw the doc correction describes, reproduced in code one commit later. RX_VK_FSR=1 now brings the function in on its own, keyed by a list of its own (RX_VK_UPSCALE_KEYS, default the gameplay network) rather than by the stand-in's conditional default, and the stand-in only runs when it was asked for. Builds clean with RX_VK_ONLY=1, 0 warnings. The camera publish needs a full game build to take effect, since it is lifted C and a host global. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two inputs the game never hands its upscaler, assembled instead of taken. **The depth.** targets() now keeps the depth target bound with a colour target, the largest one seen, so a shadow map or a quarter-resolution effect pass does not displace the one the scene was drawn with. That is the only depth there is: the gameplay network samples one colour texture and nothing else. **The motion.** gpu/vk/shaders/camera_motion.comp reconstructs each pixel's world position from the reversed device depth with the inverse of this frame's view projection, projects it with the previous frame's, and writes where it was in render pixels, which is what FSR 3.1 is compiled for (LOW_RESOLUTION_MOTION_VECTORS=1, unjittered). Far-plane pixels, a degenerate reconstruction and anything behind the previous camera all get zero motion rather than a vector from nonsense. This is camera-only by construction. It is right for a world that does not move, and a character, foliage or a particle ghosts, because its own motion is not in the depth buffer. That belongs in whatever setting turns it on, not only in a comment. The host side makes the pipeline and an R16G16F target, inverts the matrix in doubles and refuses a singular one, and stands this frame's matrix in for the previous one on the first frame and after a reset, which makes the motion exactly zero rather than arbitrary. **What is still missing is the matrix.** sead's projection update gives the projection alone; a temporal upscaler reprojects with view * projection. Rather than guess which offset of which object holds the view, nx2/oot_camera.py now also publishes the guest addresses of the projection object and of the 4x4 it writes, and HOS_CAMERA_DUMP=1 logs a window of floats around both so the matrix can be identified from what it contains. Guest addresses only. The seam's log also reports the camera it was given and the scene depth it found, so one run says whether both arrived. Builds clean with RX_VK_ONLY=1, 0 warnings; the shader compiles and passes spirv-val. Nothing calls the motion pass yet. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured before any of this changes anything, because it decides whether an upscaler is worth having here. Linux, RTX 5060 Ti, driver 615.71.09, headless under Xvfb with a warmed shader cache, the opening and Kokiri Forest: 57-60 fps, held over 33,600 presents in one run and 9,000 in another 1106 draws (0 skipped), 99 dispatches, 148 passes a frame 1536x864 internal, resolved to a 2048x1152 scene, presented at 2560x1440 Ocarina already runs at its target frame rate. An upscaler is not what gets it there, and the argument for FSR or DLSS in this title is image quality at a given cost, or headroom in areas this run did not reach -- not frames. Anyone reading this document to decide whether to spend time on it should see that first. 1106 draws and 148 passes a frame at 60 fps is also not an upscaler-shaped problem, which is worth saying where someone is choosing what to work on. Docs only. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The baseline section said Ocarina already ran at its target frame rate, which misstates the objective. 60 is the floor. The limiter at mod + 0xfc takes any rate from 40 up when variable refresh is on, so HOS_FPS_TARGET can ask for more, and the real-time frame step at main+0x6e6cc4 keeps game speed honest at whatever comes out. What an upscaler buys against a floor is the two things a floor does not give: a higher internal resolution at the same rate, and the headroom to hold a higher rate at all. The scene is 1536x864 for a 2560x1440 present, so there is a lot of resolution being left behind, and that is the gap FSR or DLSS closes. Docs only. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The seam now builds the inputs rather than looking for them in a dispatch that does not carry them: the scene depth targets() kept, and motion reprojected from it by the camera-motion pass. Only where the depth's extent matches the network's input, so a shadow map or an effect pass cannot stand in for the buffer the scene was drawn with. The Request is rebuilt for the assembled set before temporal_possible() sees it, because the gate is about what a backend will be handed and that is no longer the dispatch's own bindings. Motion scale is 1 because the pass already writes render pixels. It stays off for one honest reason, which is the last one: hos_oot_view_projection answers 0. sead's projection update gives the projection alone and the view is not identified yet, so reprojecting would be through a matrix that is not the one the depth came from. HOS_CAMERA_DUMP=1 is how that gets found; until it is, camera motion does not run and FSR falls through, rather than producing a plausible picture built on the wrong matrix. Builds clean with RX_VK_ONLY=1, 0 warnings. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run reached gameplay and published nothing. The reason is in oot_wide.py's own comment: sead's perspective projection matrix update has "three inlined copies", and the path that actually runs is one of them, not the out-of-line function I patched. So the publish now goes at all four sites oot_wide patches for the camera ratio, with the projection object taken from the register each copy holds it in -- x0, x21, x0 and x2, which are the second argument of oot_wide's _ratio() at the same offsets. The 4x4's address is published only from the out-of-line copy, where x1 was verified to hold it by reading the function. The dump dereferences that address, so publishing a guessed one from an inlined copy could fault rather than merely log nonsense. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run with HOS_FPS_TARGET=120 presents a locked 60 and logs "frame limiter 60" at every sample, 300 presents apart. The log reads +0xfc before the raise, so reading 60 every time means the engine writes it back between samples: with variable refresh on, the adaptive controller owns that word, exactly as the comment above it says. HOS_FPS_FORCE=1 also raises the three controller words that sample reads as 60 (+0x270+0x58, +0x28, +0x2c) and the nominal rate at +0x2c8, to find out whether the rate the controller settles on is what caps the frame or whether something else does. Labelled an experiment and off by default on purpose: it writes engine fields every frame whose meaning is inferred from one log line, and the real-time frame step already derives game speed from +0x2c8, so forcing that word changes two things at once. If it turns out to be the cap, the fix belongs at whatever writes the controller rather than in a host loop fighting it. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**The projection object is transient.** Publishing its address and reading it a frame later came back all zeros, while the publish itself had validated a sane near plane, far plane and field of view from the same object. The inlined copies build it on the stack, so a pointer kept for later is a pointer to nothing. The publish now copies the first 48 floats at publish time, which is the only honest way to see what is in it, and HOS_CAMERA_DUMP=1 logs that window. The view half of the view projection gets identified from those values rather than from a guess. **And the frame limiter's gate is found.** HOS_FPS_FORCE raising the three controller words and the nominal rate to 120 took -- the sample reads "nominal 120, controller 120 120 120" -- and the frame rate did not move: 60.0 fps, with the limiter still reading 60. So those words are not the cap. f_804fc0f0 is what stores a rate into +0xfc, and reading it gives three conditions: a mode must be 1, the rate must be 40 or more, and the byte at +0x232 must be set. The sample prints that byte as the second half of "vrr %d %d", and it reads 1 0: +0x231 is on, +0x232 is off, so the store never runs and +0xfc keeps its 60 whatever else is set. HOS_FPS_FORCE now sets that byte, so the game's own code can take a higher rate. Still an experiment and still off by default: it writes engine fields whose meaning comes from reading one function and one log line. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three experiments to find this, each ruling something out. Raising the limiter at +0xfc with HOS_FPS_TARGET does not stick: the sample reads "frame limiter 60" every time, 300 presents apart, and the log reads that word before the raise, so the game writes it back in between. Raising the engine's controller words and the nominal rate as well took -- "nominal 120, controller 120 120 120" -- and the frame rate did not move: 60.0 fps, limiter still 60. Setting the byte at +0x232 that f_804fc0f0 requires did not stick either; it reads 0 at every sample. f_804fc0f0 is what stores a rate into +0xfc, under three conditions: a mode is 1, the rate is 40 or more, and that byte is set. The game maintains the byte at 0, and it does that because it believes the display cannot do better. Two of the calls it asks with were unimplemented imports -- nn::vi:: SetDisplay120HzRefreshRateCapability and QueryDisplayCurrentRefreshRate -- and the third, QueryDisplayVrrSupported, was implemented and answered 0. So this is not something to force from outside. HOS_DISPLAY_VRR=1 answers QueryDisplayVrrSupported with 1 and implements SetDisplay120HzRefreshRateCapability as the acceptance it wants, and the game's own limiter code can then take a rate of 40 or more by itself. Off by default, because it changes what the game believes about the display: on a 60 Hz panel a game pacing itself faster gains nothing and may tear. The real-time frame step at main+0x6e6cc4 keeps game speed honest at whatever rate comes out. QueryDisplayCurrentRefreshRate is left unimplemented until its RefreshRate encoding is known from something better than a guess. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… matrix A run confirms the field layout and fills in the numbers. The projection object is built on the stack by the inlined copies, so a published pointer read a frame later is all zeros; a snapshot of its first 48 floats at publish time gives +0x98 = 1.0 the near plane +0x9c = 20000 the far plane +0xac = 0.41421 tan(22.5 deg), so the field of view is 45 degrees +0xa0 = 0.7854 that same angle in radians +0xb0 = 0.5625 1080/1920, so height over width, not the other way round +0xa4, +0xa8 sin and cos of the half angle which is the layout the previous commit inferred from what the code does with each offset, now with values behind it. The 45 degrees is also the needle for the rest. A projection built from it has m11 = 1/tan(22.5 deg) = 2.41421, so the view projection the draws actually use can be found by looking for that number in the uniform buffer a scene draw reads -- the matrix the depth was rendered with -- instead of guessing at another object's offsets. Also records that the path which runs is an inlined copy, since patching only the out-of-line function published nothing. Docs only. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The frame rate chase ends here. The host signals the display's vsync event from
its own thread at
const int64_t period = 1000000000LL / 60;
and the game waits on that event, so a hardcoded 60 there is a hard 60 fps
whatever else is set. The measurements say so plainly: with the limiter
unlocked to 120 by HOS_DISPLAY_VRR, on a 164.91 Hz monitor, 298 of 300 frames
still landed in the 15.5-17.8 ms window with no variance at all. Under Xvfb the
same build spread across the buckets, because there is no real display to pace
against -- which is why this did not show until a run on the actual screen.
Worth noting what each step ruled out, since three of them were wrong turns:
raising the limiter at +0xfc does not stick, raising the engine's controller
words and nominal rate changes nothing, and the gate byte at +0x232 is
maintained by the game. The limiter work was still needed -- the game clamps
itself without it -- but it was never the thing pacing the frames.
HOS_VSYNC_HZ=<hz> sets the rate, 60 by default, clamped to 15..1000: below 15
the game's own frame step treats a step as a hitch, and a rate the machine
cannot render only spends the time sleeping less. The file's header comment
said "signaled at 60 Hz by a host thread" all along.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things the hardcoded rate got wrong, both as asked. **It now follows the display.** hos_sdl_refresh_hz() reads the refresh rate out of the same SDL_DisplayMode the window size already comes from -- a float at offset 20 in SDL3, an int after w and h in SDL2 -- and the vsync thread uses it when HOS_VSYNC_HZ is unset. A hardcoded 60 paces a 164.91 Hz screen at 60 whatever else is set, which is what the measurements kept showing. **And HOS_VSYNC_HZ=0 turns the pacing off**, by signalling at the clamp's maximum so the game is not gated on this event at all. The log says which of the three it took and why, because "60" with no reason attached is what made this take four runs to find. Honest about what is not yet explained: a run with the event at 120 Hz, the limiter at 120 and the gate open still presented exactly 60.0 fps, 297 of 300 frames in the 15.5-17.8 ms window. So something past this still paces at 60 -- a swap interval, or the game presenting on every other event. The event rate had to be settable before that could be told apart from it, and now it is. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…oad at all
hos_oot_camera and hos_oot_view_projection live in the game executable, not in
this library, and I declared them as plain externs. The renderer is dlopen'd
with RTLD_NOW, which resolves every symbol up front, so the whole library
stopped loading:
dlopen failed: hookshot_vk.so: undefined symbol: hos_oot_camera
and the game fell back to "no WebGPU renderer in the Linux build: nothing is
drawn". Every run since that commit drew nothing. The frame-rate measurements I
took in between -- the vsync and limiter experiments -- were of a game with no
renderer, and none of them say anything about frame time. The mechanisms those
commits describe were read out of the code and still hold; the numbers attached
to them do not, and the one measurement that stands is the earlier 57-60 fps
with 1106 draws a frame, taken when the renderer was loaded.
Weak and null-checked at every call site: the renderer loads either way and
simply has no camera when the host does not export one, which is also the
honest behaviour on another game version where the patch sites are absent.
Verified by dlopen directly rather than by another run: the library now loads
and resolves mkwvk_init.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… holds The camera publishes tan(fovy/2) = 0.41421, so the projection's m11 is 1/0.41421 = 2.41421, and a row-major 4x4 keeps m11 five floats in. This scans each uniform buffer a draw binds for that number and logs the stage, the slot, the byte offset and the sixteen floats starting five back, which is the candidate matrix. Whatever it finds is the matrix the draw itself used, which is the one the depth was rendered with -- better than guessing at another object's offsets, which is how the projection fields were found and which only worked because the function showed what it did with each one. Off by default, reads at most 2 KB per bound buffer, stops after eight hits, and reads through ub_read, which is what the replay rules require: the submitted uniform data, not live guest memory. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… on screen Reported from a real run: the game went into slow motion. The cause is this block writing +0x2c8. The real-time frame step owns that word and the engine steps game time by 60 / it, so forcing it to 165 while the game renders at 57 advances time by 0.36 of a frame each frame -- about a third speed. The commit that added the block had already written that it "changes two things at once" and left it in anyway. It now raises only the three controller words. Also worth recording against those earlier experiments: they were all run with no renderer loaded (the weak-symbol fix explains why), so the "exactly 60.0 fps, 301 of 301 frames in one bucket" they produced was a game drawing nothing. With the renderer loaded on the same display the figure is 56.8-57.8 fps with the intervals spread across 15.5-22 ms, which is performance, not a cap. The frame limiter was never what held the rate down once anything was being drawn. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The NVIDIA half of the backend pair, written against the FSR one's shape: supported(), prepare(), upscale() that refuses rather than guesses, output_image(), shutdown(). **Licensing: nothing of NVIDIA's is in the tree.** RX_VK_DLSS_SDK is off by default; behind it NVIDIA's own headers declare the API and the compiler checks every line, and in front of it there is nothing of theirs at all. The alternative -- hand-writing the ABI so no SDK is needed -- was weighed and rejected for a specific reason: NGX is not a handful of entry points. NVSDK_NGX_Parameter is a C++ class of pure-virtual overloads whose vtable order IS the ABI, the resource and create structures are layouts, and the parameter names are strings. None of that is derivable from exported symbol names, none is compiler-checkable, and a wrong vtable slot is memory corruption inside NVIDIA's runtime on a user's machine, reported as a driver crash. That is worse than DLSS being unavailable. Even behind the gate the entry points are resolved by name at run time, so a gated build has no link dependency on NVIDIA's library either, and on a machine without the driver every pointer is null and supported() is false. Everything vendor-neutral -- the output image, the extents, the layout transition, the barriers and every refusal clause -- is outside the gate and compiled always. The SDK's CreateFeature/EvaluateFeature helper macros are deliberately not used: they name those symbols in the object file and hand the link back to NVIDIA's library. The parameters are set with the SDK's own key macros instead, so a wrong name is still a compile error, and the full key set is tabulated in docs/upscaling.md to be reviewed against the helper rather than trusted. Also fixes a real bug in my FSR output image, caught in this review: it asked for TRANSFER_DST while the seam blits *from* it, which wants TRANSFER_SRC. Untested beyond compiling, and the claim is exactly that. No DLSS runtime and no NVIDIA SDK are on this machine, so it has never run. The gated arm was syntax-checked against a stand-in header set kept outside the repository, and that check was confirmed non-vacuous. The library names are a guess, and Init_with_ProjectID's argument list varies by SDK version. Nothing calls this yet, the device enables none of NGX's required extensions, and the caller cannot fill the image and format fields NGX needs because colour arrives as a view alone. tests/test_dlss_contract.py drives the real translation unit against stub symbols and asserts the no-SDK contract and three source-level licensing properties. It was checked against six planted faults, each caught by the assertion aimed at it. It needs clang++, because gpu/render/rx_nvn.h uses C's _Static_assert, which GCC rejects in C++. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RX_VK_VP_HUNT=1 found the needle on its first run:
vp hunt: stage 5 slot 0, m11 at +68, matrix from +48:
1.358 0 0 0
0 2.414 0 0
0 0 -1 -0.2
0 0 -1 0
m11 is 2.414 as the 45 degree field of view predicts, 2.414 / 1.358 is 1.777 =
16/9, and the last row makes w = -z. The scene draws read their projection from
stage 5, slot 0, byte 48 of a uniform buffer, and it agrees with the camera the
lifted code publishes -- two independent sources now say the same thing.
It is the projection alone: the upper-left 3x3 came back diagonal, with no
rotation or translation. A temporal upscaler reprojects with view times
projection, so the view is still to be found, and it should be in the same
buffer because a draw needs both. The hunt now dumps that whole buffer in rows
of four, which is how a 4x4 reads, instead of sixteen floats around the hit,
and stops after one buffer since one dump is the point.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hader
The hunt's wider dump gave the whole buffer, 192 bytes at stage 5 slot 0:
+ 0: 1 0 0 -0 the view, 3x4 row major
+ 16: 0 0.9487 -0.3162 -0.4743 (orthonormal: 0.9487^2 + 0.3162^2 = 1)
+ 32: 0 0.3162 0.9487 -3.32
+ 48: 1.358 0 0 0 the projection, 4x4
+ 64: 0 2.414 0 0
+ 80: 0 0 -1 -0.2
+ 96: 0 0 -1 0
+112: 80 45 0.0125 0.02222 the viewport and its reciprocals
so the renderer composes projection times view itself, with the view's implied
fourth row, and that is the matrix a reprojection needs: the one the depth was
actually rendered with, not one read out of a game object a frame later.
The buffer is recognized by its contents rather than by a program key -- the
projection's m11 is 1/tan(fovy/2), its last row makes w = -z, and the rows that
should be zero are -- so a buffer that does not look like a projection is left
alone. Read through ub_read, which is the submitted uniform data and not live
guest memory, as the replay rules require.
Also fixes a bug that would have been invisible: GLSL loads a mat4 column by
column and everything here is row major, so both matrices now go into the push
constants transposed. Untransposed it does not fail, it reprojects through the
transpose, which looks like plausible motion pointing the wrong way.
Builds clean with RX_VK_ONLY=1.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
## [2.0.12](v2.0.11...v2.0.12) (2026-11-09) ### Features * a timed span per FSR pass, so the cost has a number per pass ([1d7f262](1d7f262)) * add a Quit option to the gameplay menu ([0b02149](0b02149)) * **audio:** Audio output in the Insert menu: Auto, Stereo or 5.1 ([4856258](4856258)), closes [NotRexed#1182](https://github.com/notquiteog/Hookshot/issues/1182) * census the render target sets, to settle whether a G-buffer exists ([8f2fda3](8f2fda3)) * compile AMD's FSR 3.1 upscaler passes from the FidelityFX SDK ([37afa2e](37afa2e)), closes [NotRexed#577](https://github.com/notquiteog/Hookshot/issues/577) * compose the scene's view projection, and transpose it for the shader ([df82d90](df82d90)) * count FSR evaluations, because nine passes a call is cheap or ruinous ([85a3ea2](85a3ea2)) * create FSR 3.1's pipelines and internal images ([6b51433](6b51433)) * derive camera motion from the scene's depth, and keep that depth ([29c7cbe](29c7cbe)) * DLSS at the seam, and a stage switch for the FSR cost ([6aa9f48](6aa9f48)) * DLSS Super Resolution through NGX, behind a build-time SDK gate ([aff6abb](aff6abb)) * frame generation asks for its own extensions and the whole input set ([ad291d1](ad291d1)) * FSR 3 frame interpolation, and a jitter for the upscalers to undo ([384c41b](384c41b)) * give FSR the assembled depth and camera motion ([481b957](481b957)) * HOS_DISPLAY_VRR, which is what actually holds the frame rate at 60 ([c65c68e](c65c68e)) * HOS_FPS_FORCE, because raising the limiter alone does not hold ([2008ac7](2008ac7)) * HOS_VSYNC_HZ, the hardcoded 60 that actually paces the game ([5eb9f8b](5eb9f8b)) * link NGX properly, frame generation at the seam, AMD's interpolation sources ([43e3360](43e3360)) * Monitor's shape for screens narrower than 16:9 (4:3, 16:10, 5:4) ([b44c9cd](b44c9cd)) * name each network binding's role in the upscaler input contract ([d9566a1](d9566a1)) * publish Ocarina's camera, and make a backend reachable without the stand-in ([8bd7c8d](8bd7c8d)) * record FSR 3.1's nine dispatches ([8769c80](8769c80)) * route the game's upscale dispatch through one backend seam ([1a036c8](1a036c8)) * RX_VK_VP_HUNT, to find the view projection from a value only it holds ([3c02213](3c02213)) * the hunt found the projection, so widen it to find the view ([2946a32](2946a32)) * the parallel lock says which site took it, and the answer is two sites ([b8543c9](b8543c9)) * the two backends have rows in the Insert menu ([cf8b3b1](cf8b3b1)) * the two lock sites say why they had to take it ([614f552](614f552)) * the upscaler and frame generation follow the menu while you play ([61d7d4a](61d7d4a)) * the upscalers and frame generation are the native settings, not new rows ([a0ce7de](a0ce7de)) * the vsync event follows the display, and 0 turns it off ([a67e471](a67e471)) * **vk:** AMD FSR 4 upscaling and frame generation through AMD's runtime ([65003cc](65003cc)) * **vk:** choose the frame generation method in the Insert menu ([c4f7d84](c4f7d84)) * **vk:** DLSS 5 neural rendering through NGX directly, beside Streamline ([7784399](7784399)) * **vk:** DLSS 5 neural rendering, with the model's own settings in the menu ([2851bc6](2851bc6)) * **vk:** frame generation Auto: a frame for each of the display's refreshes ([a5a7440](a5a7440)) * **vk:** say why DLSS 5 neural rendering is refused, and the RTX 20-40 route ([60f1d2e](60f1d2e)) * **vk:** time a capture's GPU wait for its late pages apart in the log ([e3b622f](e3b622f)) ### Bug Fixes * a fork releases its own work, not upstream's ([47dd097](47dd097)) * a Windows build could ask for DLSS and then fail to link ([d72441a](d72441a)) * **audio:** 5.1 surround when the default output has it ([NotRexed#1124](https://github.com/notquiteog/Hookshot/issues/1124)) ([b0abeca](b0abeca)) * **audio:** a finished wave buffer reaches the game at the renderer update, not mid-pass ([28e4a11](28e4a11)) * **build:** a Windows build linked the ARM64 NGX library, not the x64 one ([70b7d25](70b7d25)) * **builder:** copies from a read-only install stay writable ([70ffa5e](70ffa5e)) * **builder:** stop when generated source cleanup fails ([d254e6f](d254e6f)) * **builder:** the window icon is copied writable too ([4b2d871](4b2d871)) * **build:** NGX asks for Microsoft's C++ runtime, which llvm-mingw has not ([114e917](114e917)) * **build:** the feature announcement swallowed the whole Vulkan build ([1004f37](1004f37)) * **build:** the twelve MSVC runtime symbols NGX asks of an llvm-mingw link ([d00159a](d00159a)) * clear the SPD counter between the pyramid passes (did not fix the cost) ([66e8c3a](66e8c3a)) * contain pipeline cache save allocation failures ([4518a20](4518a20)), closes [NotRexed#1022](https://github.com/notquiteog/Hookshot/issues/1022) * HOS_FPS_FORCE must not touch the nominal rate: it is slow motion on screen ([8681f2e](8681f2e)) * **lift:** clear an earlier lift's hot pieces before the patches; count every made and raw output-size place ([816a472](816a472)), closes [NotRexed#912](https://github.com/notquiteog/Hookshot/issues/912) * **lift:** edit_c in chunks.py, so its tests import without the lift's tools ([1c5258f](1c5258f)) * **lift:** group page marks after the patches; a missing output-size edit stops the lift ([3fa6300](3fa6300)) * **linux:** the AppImage's parts list names the repository it is built in ([2228816](2228816)) * narrow screens: window at the picture's shape, save picture's middle band, comments ([3060261](3060261)), closes [NotRexed#1183](https://github.com/notquiteog/Hookshot/issues/1183) * preserve NUP member names during implicit reads ([1fdfeff](1fdfeff)) * publish the camera from the inlined projection copies too ([c630d59](c630d59)) * read the upscaler contract in descriptor order, not from draw_sampled ([492bcd0](492bcd0)) * **release:** the fork's releases follow upstream's v2.0.x line ([d495994](d495994)) * **release:** the setup, the Builder and the game update from the repository they were built in ([6fd4364](6fd4364)) * **release:** the Windows release installs the renderer CI built, with DLSS, NR and FSR 4 ([0fa18bd](0fa18bd)) * scan for the view projection once a target set, not once a draw ([12d4d83](12d4d83)) * snapshot the projection's fields, and name what caps the frame rate ([3790556](3790556)) * the conventionalcommits preset has to match the changelog writer ([ca6595f](ca6595f)) * the DLSS contract test links again ([86b6988](86b6988)) * the game's network takes one texture, so it is not a temporal upscaler ([853cfe3](853cfe3)) * the history reset the document specifies, which nothing implemented ([9858599](9858599)) * the host camera symbols must be weak, or the renderer does not load at all ([70ecdf4](70ecdf4)) * the release's commit-analyzer preset is a package, and it was not declared ([3ff9fbd](3ff9fbd)) * the renderer would not load, and the build now says so ([c871a3c](c871a3c)) * the shader check compared glslang's version, and ran where glslang is not ([f48483c](f48483c)) * the view projection scan ran per draw, which stuttered hard ([2d9b4e8](2d9b4e8)) * **vk:** a draw that waits for its pipeline runs the build at once if it is still queued ([aafbaf5](aafbaf5)) * **vk:** DLSS frame generation sets NGX's parameters through its C functions ([2c19b63](2c19b63)) * **vk:** frame generation Auto: the snap distance is not named near (windows.h defines it empty) ([d156a6d](d156a6d)) * **vk:** keep the last sRGB UNORM surface format, so drivers that list only sRGB keep theirs ([3408437](3408437)) * **vk:** Linux/Wayland: take an sRGB surface format for the swapchain ([7416487](7416487)) * **vk:** no mirror is made inside a parallel replay's segment (an earlier segment read it unfilled) ([8f0ed68](8f0ed68)) * **vk:** tag the uplift colour pair as whole resources, as the shipping integration does ([dd3e61e](dd3e61e)) * **vulkan:** bind only bytes accessed by buffer-image copies ([b0a9034](b0a9034)) * **vulkan:** clean up failed mesh argument initialization ([b2f5692](b2f5692)) * **vulkan:** preserve guest mip levels in partial clears ([d7a1207](d7a1207)) * **vulkan:** release modules from rejected cache records ([2838e89](2838e89)) * **widescreen:** the save picture shows the middle 16:9 on ultrawide ([NotRexed#952](https://github.com/notquiteog/Hookshot/issues/952)) ([e2e155e](e2e155e)) ### Performance Improvements * **fpx:** keep the hardware result for FMUL/FMLA S when the flags are already set ([6de3bf1](6de3bf1)) * **lift:** one page mark per straight-line group of stores through a base register ([825861b](825861b)) * **lift:** the interrupt poll before each backward goto, calling a64_poll_cur() with no argument, so a loop's guest registers stay in host registers ([0d6ed03](0d6ed03)) * stream sampled profiles instead of loading every line ([8cf6d1e](8cf6d1e))
|
We can support NR with NGX, which is cross-platform. Let's do that None of DLSS is open-source, so it's moot |
…etic data There are no real weights to test with, so scripts/opennr_harness.py builds upstream's own dlss5vk tool from the vendored sources with the renderer's compiler, shim and portability patch, gives it the kernels the renderer ships, a synthetic model directory in exactly upstream's manifest layout (153 records, 11 stage files, SHA-256s, scaled so the activations stay alive through all 71 blocks, which it checks) and a synthetic frame, and lets upstream's own checkers give the verdicts: - the GLSL reference route (unfused, every PTX family off, the PTX folder empty) is the reference for all 75 boundaries and the head; - upstream's parity runs every other route against it; - upstream's verify runs block 0 against the CPU reference, built in two models of the tensor cores: upstream's (Ada: FP8 k32 as two groups of 16, F13; FP16 k16 as two of 8, F24) and one group per MMA step (FP8 32 products, FP16 16), which is what this Blackwell card matches exactly while it differs from the Ada model by an ulp in a quarter to a half of every tensor-core kernel's values. tests/test_opennr_harness.py is its quick form (384x256, ~20 s); it says SKIP without the GPU, Vulkan, a compiler or the headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…S-NR A third backend behind the Neural tab: the vendored OpenDLSS-NR network on the renderer's own device, with no NVIDIA runtime, so it runs natively on Linux. Selected with RX_VK_NR_BACKEND=open for now; the one MERGE line in vk_opennr.cpp is where it becomes the Backend row's fourth value. - The weights are the player's: a model directory (RX_VK_OPENNR_MODEL, else nr-model in the game folder) checked as upstream's loader checks it (stage lengths and SHA-256, tensors in range, 71 blocks) on a thread of its own. Without one the log and the row's hint say so and frames go out untouched. Nothing here extracts or fetches model data. - Same place and contract as the Streamline backend: dlssnr::apply hands it the finished frame in both of its builds; the headless present path calls it too, so a headless run exercises it. - The frame: a copy of the window's image, opennr_pre.comp (the 16 input lanes), the network pre-recorded once per size in a secondary command buffer, opennr_post.comp (residual, temporal blend with the model's blend_scale, history, style, intensity). GENERAL layouts, whole-pipeline barriers around the network, nothing per frame. - Temporal, with the scene's camera-only motion; RX_VK_OPENNR_TEMPORAL=0 is single-frame. Fast route (PTX) when VK_NV_cuda_kernel_launch is there, the GLSL kernels alone otherwise; a timed-out chained wait falls back to barriers, as upstream's demo does. - vk_main.cpp asks for coopmat2, float8, CUDA launch and SM builtins only when the backend is selected, and logs what is missing. - nx2/build.py compiles upstream's host sources from a patched copy in gen and embeds the generated PTX. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hckrman101
left a comment
There was a problem hiding this comment.
Thanks for this. There's real work in here, especially the FSR 3.1 frame interpolation, which no other PR has. I read the hand-written integration against current main and skipped the vendored SDK sources. Provenance and licence notes for third_party/ffx-api and third_party/fidelityfx-sdk look fine: MIT, pinned commits, unmodified.
I agree with the split NotRexed asked for. Below is what I found, so each piece can land cleanly.
Bugs in the FSR 3.1 upscaler (gpu/vk/vk_fsr3.cpp, vk_draw.cpp seam)
- Resize leaves the new images uninitialised.
prepare()remakes every internal image when the render or output size changes, butfirst_evaluationis only reset byshutdown(). On the nextupscale(),firstis false, so the new images never get their UNDEFINED→GENERAL barrier or their clear.lut_uploadedalso stays true, so the newlanczos_lutis never filled. After any size change (window resize, Render resolution, a dynamic-resolution change) the passes run on undefined layouts and an empty Lanczos LUT. - Resize frees images the GPU may still be using.
make_image()→drop_image()destroys the old images straight away, with no idle and no deferred free, while earlier command buffers can still reference them.vk_dlssnr.cppalready does this correctly withdevice_wait_idle_locked()underpresent_mu.vk_ffx.cpp'sidle()uses a barevkDeviceWaitIdlefrom the replay thread. That breaks the external-sync rule your own comment invk_dlssnr.cppdescribes, because the present queue can be submitting at the same moment. - The blit isn't synchronised with the output write. The last barrier in
fsr3::upscale()has dstFRAGMENT|COMPUTE / SHADER_READ, but the seam then readsfsr3::output_image()withvkCmdBlitImage, which isTRANSFER / TRANSFER_READ. The blit isn't ordered after the RCAS/accumulate write. Nothing orders the blit's write to the game's target before the game's next reader either. Separately, B10G11R11 isn't guaranteedBLIT_DST, so the format feature should be checked once. - Colour space. The network's input is the square root of the tonemapped colour, and main's DLSS path (#841,
dlss_out.comp) squares the result back to linear before writing the network's output. This PR gives FSR 3.1 and FSR 4dw().imgs[0]/dw().srcs[0]directly and blits the result into the target without squaring. If I'm reading main right, the FSR picture comes out in the wrong transfer function. It also saysFFX_UPSCALE_ENABLE_HIGH_DYNAMIC_RANGEfor an input that isn't linear HDR.
Duplicate inputs. nx2/oot_camera.py plus vk_camera_motion.cpp add a second camera hook and a camera-only motion pass, but main already has #841's jitter (nx2/oot_jitter.py) and motion vectors with the skinned actors' own motion. oot_camera.py also changes the lifted C for everyone, at four projection sites every frame. #1008's vk_temporal.cpp moves #841's chain behind an engine interface. FSR 3.1 should plug in there as one more engine, so there's one jitter in the game and FSR gets actor motion too.
Upscaler values clash with #1008. Both PRs extend the saved dlss key past DLAA with different meanings: here 3 = FSR 3.1 and 4 = FSR 4, in #1008 3 = FSR 4 and 4 = FSR 4 Native AA. Whichever lands second has to renumber. I'd suggest keeping #1008's 3/4 and adding FSR 3.1 as 5, so nobody's saved ini changes meaning between releases.
Teardown. None of the new shutdown() functions (ffx, fsr3, fsr3fi, dlssnr, dlssnr_ngx, streamline, dlssg, camera_motion) is called anywhere. That avoids the exit-time ffxDestroyContext-from-a-static crash we found in #1008, but only because nothing is ever destroyed. If teardown gets wired up later, it needs to run before the device goes and never from a static destructor.
DLSS 5 NR. This isn't ray reconstruction. It's NVIDIA's "neural rendering" uplift, run over the finished frame before the HUD. It worries me most for main:
- The Streamline path's
DlssNrOptions(GUID, struct version, layout) is reverse-engineered from a prerelease DLL, because the real header is only in NVIDIA's NDA SDK. The NGX-direct path usesNVSDK_NGX_Feature_Reserved18with guessed parameter names. - readme and
explain_refusaltell RTX 20–40 users to use a patchednvngx_dlssnr.dlland setHKLM\...\NGXCore\EnableSignatureOverride=1. That turns off NGX signature checking for every game on the PC. I don't think upstream should document or support that. - Per the thread, NR should go through NGX, cross-platform. Brett-D's path uses NVIDIA's own NR headers and needs neither Streamline nor the reconstructed struct, which looks like the better base.
Release and CI. These need to come out of the feature PRs:
--accept-nvidia-licensein both workflows, and the prebuilthookshot_vk.dllshipped in the Windows package with NGX statically linked. That changes main's "each player accepts NVIDIA's licence in the Builder" model, and it's the maintainer's call.Repository.cs, theHOOKSHOT_REPOSITORYplumbing insetup.c,hos_update.c,linux_update.pyandversion.py,.releaserc.json(feat→ patch),changelog.txtand the fork's version bumps. That's fork release machinery.prebuilt_renderer()inmake_game.py: the Builder stops compiling the renderer on the player's PC.
Unrelated behaviour changes that should be their own PRs:
nn_vi.c: the vsync event now follows the display's refresh rate by default instead of 60. That changes pacing for everyone on a high-refresh monitor, with no opt-in.HOS_DISPLAY_VRR,HOS_FPS_FORCEand the 120 Hz HLE.- The ParLock per-site counters, plus the always-on atomic counters in
texture_touch_quickandgpu_buffer_bind, which sit on hot paths.
Linux. FSR 4 (vk_ffx) and Streamline are Windows-only stubs, and FSR 3.1 is the only new path that compiles on Linux. CI never ran on this PR (head is the fork's main, and there are no checks), so there's no evidence either way that the Linux build or tests/run_all.py pass. It also conflicts with main (39 commits behind).
On #1008 vs this: I'd take #1008's architecture as the base for main: the TemporalEngine on #841's inputs, the right colour handling, and it runs on Linux. That makes #1008 the FSR 4 path, so vk_ffx.cpp here would duplicate it. From this PR, the unique and valuable parts are the FSR 3.1 upscaler, as an engine on vk_temporal (with the fixes above), and FSR 3.1 frame interpolation (vk_fsr3fi.cpp) plus the Frame generation method row. Those are a great first split PR. NR should come separately over NGX.
| # what the release's Builder installs (nx2/make_game.py prebuilt_renderer): a renderer compiled on the player's | ||
| # PC has none of NVIDIA's SDKs, and the Insert menu then offers none of DLSS, its frame generation or NR. | ||
| - name: NVIDIA's DLSS SDK (headers and nvsdk_ngx_s.lib) | ||
| run: python -I scripts/ngx_sdk.py --platform windows --build-only --accept-nvidia-license "$RUNNER_TEMP/ngx-sdk" |
There was a problem hiding this comment.
This accepts NVIDIA's DLSS licence on every release build and links NGX into the shipped hookshot_vk.dll. Main's model is that each player accepts it in the Builder, and changing that is the maintainer's call, so please leave it out of the feature PRs.
| # (libnvidia-ngx-dlss.so) is NVIDIA's to ship and is still not included -- a player who wants DLSS puts it in the | ||
| # game's ngx folder, and readme.md says so. Without this the released build offers FSR and nothing else.) | ||
| - name: NVIDIA's DLSS SDK (headers and libnvsdk_ngx.a) | ||
| run: python3 -I scripts/ngx_sdk.py --platform linux --build-only --accept-nvidia-license "$RUNNER_TEMP/ngx-sdk" |
There was a problem hiding this comment.
Same here for the AppImage: --accept-nvidia-license in CI makes every release accept NVIDIA's licence. Please drop it from the feature split.
…at was not What OpenDLSS-NR is and its licence; that the model directory is the player's to supply and Hookshot provides no way to get one (the AMD paragraph no longer suggests one); how the frame is built around the network; temporal with camera-only motion, and its limits; which Neural tab rows have a counterpart in the network and which have none; what the harness measured on this Blackwell card (routes bit-exact with each other, the CPU reference matching only with one-group MMA steps, the route divergence at large synthetic activations, not root-caused) and what was not (image quality, real-frame timing); and what an AMD route needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kend The open network (OpenDLSS-NR, MIT: a Vulkan implementation of the DLSS 5 NR network) as the Backend row's fourth value, Open (OpenDLSS-NR), beside Auto, Streamline and NGX direct. It reads a model folder the player supplies (nr-model in the game folder, or RX_VK_OPENNR_MODEL); Hookshot includes no weights and no way to get them. Reconciled with the NGX-direct branch: nr::apply (vk_dlssnr_ngx.cpp) is the one dispatch point, OPEN = 3 in nr::Backend; Auto runs Streamline, then NGX direct where it can, then Open where the device was made for it. opennr::selected() now makes the device for the network when the row says Open, RX_VK_NR_BACKEND=open, or a model folder is in place at the start (a device cannot gain extensions later, and why() says to restart); ready() is Auto's test. The tests follow: the Backend row is 0..3, the open backend is reached through nr::apply, and no MERGE marker is left. The PTX generators get an absolute output folder (they run from their own). Built and linked on Linux (78 PTX kernels compiled in), 198 tests pass; Hookshot's own files of the backend pass a MinGW syntax check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…needs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## [2.0.13](v2.0.12...v2.0.13) (2026-11-09) ### Features * **vk:** open neural rendering, the DLSS 5 NR network run by OpenDLSS-NR ([1da7eff](1da7eff))
…rative_matrix alone A kernel route for GPUs without VK_NV_cooperative_matrix2 (AMD RDNA 3 and 4, llvmpipe): the four matrix kernels of upstream's unfused graph (gemm_fp8, gemm_f16, window_attend, global_attend) rewritten at subgroup scope for the MMA shapes a driver lists, with upstream's host contract unchanged, so its nr::Kernels drives them as they are. - Five variants (gpu/vk/opennr/opennr_khr.h, scripts/opennr_kernels.py KHR_VARIANTS): E4M3 16x16x32 into f16 (NVIDIA's own MMA), E4M3 into f32 at k16 (RDNA 4) or k32, and E4M3 widened to f16 into f32 at 16x16x16 (RDNA 3) or 8x8x8 at 8 lanes (llvmpipe). Every k32 step rounds to half where the network does; f32 accumulators are rounded on the bits (roundF16) after each step. - No element position inside a matrix is assumed: operands are staged with plain indexing, results go back through shared memory before any per-element work. - The f16 GEMMs of the f32 variants are integer arithmetic, the one-group model measured on Blackwell: f16 products summed in f32 double-rounded 274 of 3.9M adapter outputs. - Upstream's ops, gemm_reduce and normalize kernels are compiled from build-time copies with the hardware E4M3 conversions in integers and the half fma, rsqrt and reciprocal exact (khr_numerics.glsl); checked over every positive half that these equal what upstream's reference computes. - Checked in as gpu/vk/shaders/opennr_khr_spv.h; --check and spirv-val cover it, and the test compares the two variant tables and that no khr module names an NV extension. third_party stays byte-identical. - vk_context.cpp gets a SOURCE_PATCH: HOOKSHOT_OPENNR_SUBGROUP_SIZE replaces upstream's fixed requiredSubgroupSize 32 for the khr route only (llvmpipe has 8 lanes only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… routes cannot The renderer side of the khr kernels: opennr_net gains Route::Khr (upstream's unfused graph, every PTX family off, the kernels of shaders-khr-<variant>, pipelines made at the variant's subgroup size), and vk_opennr chooses it. - device_created() asks the driver's cooperative matrix shapes and picks the first khr variant it lists (opennr_khr.h), and logs the shapes, the variant and why not the others. Upstream's routes still run where VK_NV_cooperative_matrix2 (all seven features) and E4M3 matrices are; everywhere else with VK_KHR_cooperative_matrix the khr route does. RX_VK_OPENNR_ROUTE=khr forces it on any GPU (and then only VK_EXT_shader_float8 is asked for), RX_VK_OPENNR_KHR_VARIANT picks a variant. - Every khr variant's folder is written out beside upstream's kernels, hashed with them. - selected() is true on any GPU that is not NVIDIA's, so on an AMD card the Backend row's Auto falls to Open and the Neural tab says what Open needs (a model folder) instead of NGX's NVIDIA-only refusal; the khr route asks for nothing the renderer does not already, but E4M3 matrices where there are any. - The Neural rendering and Backend rows' help says Open runs on NVIDIA RTX 40/50 and AMD RX 7000/9000. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…llvmpipe scripts/opennr_harness.py run --khr runs every khr variant on every device a probe finds that can run it (scripts/opennr_khr_probe.cpp: the renderer's own variant table), against the references of upstream's GLSL route, each on a device made with nothing but what the khr route needs. - The tool's own device, in the harness's copy of vk_context.cpp only (TOOL_PATCHES): OPENNR_DEVICE picks a device by name; OPENNR_KHR=1 enables VK_KHR_cooperative_matrix and, unless OPENNR_KHR_FLOAT8=0 or the device has none, VK_EXT_shader_float8 -- no NV extension at all. - A khr run starts from the reference run's recorded input features (a fixture-features fixture), so a device whose log and cosine differ still runs the network on the same bytes. - Each boundary and the head are compared against the references: the share of bytes that differ, the distance in E4M3 codes, the largest difference in value (compare.txt per run); block 0 kernel by kernel against the CPU reference under a third MMA model, a finely summed (f32-like) step. - Gates: an exact variant must be bit-exact; an f32-accumulating one bit-exact, or close (block 0's kernels equal the CPU reference in all but 1e-4 of the values) on hardware whose MMA adds in another order. - Without a GPU for upstream's routes the references come from the first khr run, and say so. - tests/test_opennr_khr.py, the quick form at 384x256: SKIP when no device runs a khr variant. Measured on this PC (RTX 5060 Ti, driver 615.71.09; llvmpipe, Mesa 26.2.3 / LLVM 22.1.2): every khr variant that runs here -- e4m3-k32-f16, e4m3-k32-f32 and f16-k16-f32 on the RTX, f16-k8-f32 on llvmpipe -- is bit-exact at all 75 boundaries and the head, at 384x256 (all), 1280x720 and 2048x1152 (the RTX). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the Insert menu Reported on v2.0.13 (Windows): F1/Insert no longer opened the menu. The releases since v2.7.0 put AMD's FSR 4 runtime beside the game, and the first draw of the menu's Upscaler and Frame generation method rows asked it what it offers (upscale_supported, frames_supported): loading the loader and running ffxQuery on the menu's thread. A runtime that hangs or faults there takes the menu with it. AMD's runtime is now asked once, right after the device is made, on the render thread (ffx::probe from vk_main.cpp), with a log line before the load and before the query so a hang names itself, and every call into AMD's DLLs (load, query, create, configure, dispatch) is guarded with __try: a fault inside the runtime turns FSR 4 off rather than ending the game. The menu and the frame only read the stored answer. RX_VK_FFX=0 (hookshot_env.txt) still keeps the runtime from being loaded at all. Checked: compiles for Windows with clang's MinGW target (-fms-extensions, as CI builds it) and for Linux; 198 tests pass. Not yet confirmed to be the reporter's cause: the log will say. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FSR 3.1's interpolation has no phase input (AMD's ffx_frameinterpolation.h fixes t at 0.5): it makes the midpoint between two of the game's frames and nothing else. At 3x and above vk_fsr3fi.cpp copied that one midpoint into every made frame of the set, so the same picture was shown at 1/3 and 2/3 of the interval: judder, not frames. framegen_methods.h gains makes_set() (FSR 3.1: 2x only) and first() (the method the chain tries first for a set), and the chain passes FSR 3.1 over for 3x and larger sets, which go on to the renderer's own interpolation. The Frame generation method row's FSR 3.1 hint says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## [2.0.14](v2.0.13...v2.0.14) (2026-11-10) ### Bug Fixes * **vk:** AMD's FSR runtime is asked at the start, guarded, never from the Insert menu ([58ed3bd](58ed3bd))
## [2.0.15](v2.0.14...v2.0.15) (2026-11-10) ### Features * **vk:** the open backend runs the khr route where upstream's NVIDIA routes cannot ([5b95083](5b95083)) * **vk:** the open backend's khr kernels, OpenDLSS-NR on VK_KHR_cooperative_matrix alone ([1b80745](1b80745))
64 upstream commits. The setup's compile line keeps the fork's HOOKSHOT_REPOSITORY and takes upstream's --dependent-load-flag=0x800; the test runner keeps both sides' entries; the new stick dead zone test's fixture gains the fork's nr_backend, which ini_key names. Renderer builds and links, 213+1 tests pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## [2.0.16](v2.0.15...v2.0.16) (2026-11-10) ### Features * **linux:** the OoT debug tools on Linux (cheats, teleport, flight, free view) ([f320ecb](f320ecb)), closes [#ifdef](https://github.com/notquiteog/Hookshot/issues/ifdef) [#if](https://github.com/notquiteog/Hookshot/issues/if) * raise the Motion sensitivity X limit to 600% ([0e709de](0e709de)) * **vk:** FXAA in the gameplay TAAU stand-in (edges on cards without the network) ([84d7baf](84d7baf)) ### Bug Fixes * **builder:** check setup list names; load the setup's libraries from System32 ([05d343c](05d343c)) * **builder:** do not resolve the update staging directory itself ([25f5aaf](25f5aaf)) * **builder:** explain unavailable selected game input ([0eeae7b](0eeae7b)) * **builder:** keep Linux update extraction from writing through symlinks ([7ac1186](7ac1186)) * **builder:** stop when the working exefs cannot be cleared ([46f060a](46f060a)) * end the game when the GPU device is lost ([fe5f5ad](fe5f5ad)), closes [NotRexed#1275](https://github.com/notquiteog/Hookshot/issues/1275) * **env:** hookshot_env.txt sets only the game's own switches ([c25cb42](c25cb42)) * **fs:** keep guest paths in their mounts; do not work through links ([9a88357](9a88357)) * ignore half-float NaNs when locating flashes ([99e1867](99e1867)) * **input:** move the stick dead zone to option 11 ([ddb03fa](ddb03fa)) * **linux:** reject symlinked update staging folders ([b65d80e](b65d80e)) * **nvn:** keep host pointers in host-side tables; bounds-check guest ranges ([c19b6b7](c19b6b7)) * **runtime:** report Windows display startup failures ([eb928ad](eb928ad)) * **saves:** refuse links in a save being imported ([54865d4](54865d4)) * **sdl:** keep the mouse captured when the pointer leaves the window ([3757673](3757673)) * **update:** check the latest release's tag on Windows too ([c72e44d](c72e44d)) * **vk:** a timed-out batch of write-backs keeps its staging buffers; the sweep's budget test cannot wrap ([8a453c4](8a453c4)) * **vk:** check the shaders seed, program cache, pipeline notes and state list as read ([1ff08e5](1ff08e5)) * **vk:** send the held batch before waiting for a recording ([8794dd4](8794dd4)) * **windows:** load the Vulkan loader and SDL from known folders only ([1f02f94](1f02f94)) * write out the queued log before a fatal replay exit ([fb879db](fb879db)) ### Performance Improvements * **vk:** the retirement sweep's texture write-backs in batches, one submission and one wait each ([e89338e](e89338e))
…lour encoding (NotRexed#1386's inputs) Ports the fork onto the inputs of upstream PR NotRexed#1386, which found these bugs in the fork's FSR path: - The camera hook (nx2/oot_camera.py, host/hos_oot_camera.c) published sead's projection object (near 1, far 20000), not the projection the scene is drawn with (near 0.1, far 1000.5, fov 45), and the fork treated the scene's depth as reversed-infinite: it is 0 at the near plane. FSR's passes want reversed depth, so they are given 1 - d with factors from the real projection (gpu/vk/fsr3_math.h, checked by tests/test_fsr3_math). - FSR's output went into the network's output without undoing the network's square-root encoding; it is squared back now, as DLSS's picture is. - vk_work.h was missing from the FSR files (a barrier could be skipped), the motion image never got its first layout transition, and a resize freed images frames in flight still used. All FSR inputs are made in gpu/vk/vk_fsr3_inputs.cpp (one jitter, NotRexed#841's oot_jitter in every lift; camera from the projection; camera-only motion). FSR 4 (vk_ffx.cpp) and the frame generation chain take the same Frame. The camera hook, the fork's viewport jitter and the seam/contract headers and their tests are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## [2.0.17](v2.0.16...v2.0.17) (2026-11-10) ### Bug Fixes * **vk:** FSR 3.1 and FSR 4 take the scene's real camera, depth and colour encoding ([NotRexed#1386](https://github.com/notquiteog/Hookshot/issues/1386)'s inputs) ([366c268](366c268)) * **vk:** FSR 3.1 frame interpolation makes 2x sets only ([c719d8c](c719d8c))
host/hos_update.c defined its fallback HOOKSHOT_REPOSITORY inside #ifdef _WIN32, so a Linux compile without -DHOOKSHOT_REPOSITORY failed (nx2/build.py always passes it, so released builds were not affected). Compiles on both platforms with and without the define. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## [2.0.18](v2.0.17...v2.0.18) (2026-11-10) ### Bug Fixes * **update:** the default repository outside the Windows-only block ([ea6141a](ea6141a)), closes [#ifdef](https://github.com/notquiteog/Hookshot/issues/ifdef)
Exactly as the title says, adds AMD FSR4 and DLSS5 NR with options, for older systems it supports FSR 3.1, next update adds NR for AMD and DLSS5 NR on linux