Repository navigation
Conversation
Several open requests ask for DLSS, FSR and a proper upscaling technique (NotRexed#744, NotRexed#691, NotRexed#687, NotRexed#672), and two report the current frame generation (NotRexed#755, NotRexed#639). NotRexed#577 is adding FSR 4 Native AA on the finished image. Before more of it is written, this writes down where a temporal upscaler belongs in this renderer and in what order the pieces can land. The finding that decides it: OoT evaluates its own neural TAAU network (program fd029a317a9a4b94), which `taau_stand_in` in gpu/vk/vk_draw.cpp already recognizes and substitutes, because the translated network loses the blue channel. The game therefore already renders the 3D at a reduced internal resolution and resolves it temporally. A vendor upscaler takes that dispatch's place, with the existing stand-in as its fallback; the composited window image, where `upscale` and the frame interpolator work, has no depth, no motion and a HUD in its history. The document states the input contract such a backend must check before it runs, keeps the replay rules of design.md section 4, sets out six single-topic pull requests starting with a trace switch rather than a backend, and records what each vendor runtime permits: FidelityFX sources are MIT and need no user runtime, the Streamline headers are MIT but its libraries may never be committed or bundled, and Linux has no Streamline build so it needs NGX directly. It also says plainly that Ray Reconstruction is not implementable here: it needs albedo, normal, roughness and specular-hit-distance guides, and a search of gpu/ finds no such target in a rasterized game's replayed command stream. Docs only; no code or behaviour changes. `python -I -B tests/run_all.py` unaffected. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The log that reports a game network once per key (vk_draw.cpp) listed its sampled textures as extents and format numbers, leaving the reader to work out by hand which one a temporal upscaler would take for colour, motion and depth. Before an upscaler can be given that dispatch (docs/upscaling.md), the contract has to be read off the real thing, so this makes the log say it. gpu/vk/upscale_contract.h decides which sampled binding is which, and gpu/vk/upscale_format.h decides what a Vulkan format counts as. Motion and depth fix the render extent, since they are the two a network cannot sample at another size, and they must agree or nothing is reported: two that disagree mean the guess about which binding is which is wrong. The colour at that extent is the input. A zero extent is a binding the renderer could not resolve and is skipped rather than taken for motion. A colour at a larger extent is reported as "full-size" and nothing more. It is the network's own history, its output, or both, and draw_sampled cannot tell them apart, because it holds what a dispatch's descriptors sample *or* store in one list. Calling it the history would have been a guess printed as a fact; the seam that replaces the dispatch resolves the output image itself and can then say which it is. The contract header has no Vulkan include and reads nothing from guest memory; the format mapping is a header of its own rather than a static in vk_draw.cpp so that tests/upscale_contract_test.cpp covers it, which is the half most likely to be wrong: a format wrongly called motion is handed to an upscaler as motion. Also corrects this file's own claim from the previous commit: the compute stand-in replaces the network only where the device has no hardware f16 matrix multiply (or with RX_VK_NO_F16), and otherwise the network runs as itself. A replacement therefore cannot live inside taau_stand_in alone. No switch and no behaviour change: the extra work is one classification per network key, on a path that already builds a line once per key. Tested with python -I -B tests/run_all.py (88 passed; the 8 failures and 12 skips are the same on upstream/main here, all for a missing clang++ or SPIR-V headers), and each refusal in the classifier was confirmed to fail the test when removed. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs/upscaling.md row 2. A recognized upscale dispatch is now resolved once into an upscale::Request -- what the sampled bindings provide, the image it writes, and which binding the renderer hands over as colour -- and upscale::choose() says which backend takes it. A backend does not re-derive the dispatch for itself, which is what keeps FSR and DLSS from each growing their own copy of this. No behaviour change. The compute stand-in is the only backend, and it is a spatial resolve that needs none of the input contract, so it still runs wherever RX_VK_TAAU_KEYS asks for it, including where the contract is not met. choose() returns StandIn exactly when push descriptors are available and the key list asks for it, which is the same condition the first guard tested before, and tests/upscale_seam_test.cpp pins that: making the stand-in require the contract fails it. upscale::temporal_possible() is the gate a temporal backend has to pass. Complete inputs, an output that is resolved, writable and no smaller than the render extent, and the colour the renderer hands over being the colour the contract identified. That last clause does not hold yet, and the seam refuses rather than assuming it: the renderer passes dw.imgs[0], the first texture descriptor, while the contract indexes the textures the descriptor build resolved (draw_sampled). Those are different lists -- dw.imgs carries every texture descriptor, draw_sampled only the resolved ones -- so binding 0 of one is not binding 0 of the other. Establishing which descriptor is the contract's colour is the first job of the backend that needs it. The seam logs the refusal once per program with the output and input extents it resolved, so the next step reads what it needs from a run rather than guessing. One place now builds the binding list (upscale_bindings), so the per-network log and the seam cannot disagree about what a dispatch sampled. Tested with python -I -B tests/run_all.py: 89 passed, and the 8 failures and 12 skips are the same on upstream/main here, all a missing clang++ or SPIR-V headers. Each refusal in the gate was confirmed to fail the test when removed. The renderer itself is not compiled here: a clone carries no CPM dependency set, so the call site is reviewed against the guard it replaces rather than built. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…mpled The seam could not hand a temporal backend its colour input, because the list the contract was reading is not the list a backend is handed. draw_sampled records a sampled texture only where the GPU has written it (vk_draw.cpp, the gpu_written guard) and records every storage image as well, so its index i is not dw.imgs[i]. A backend wired to that correspondence would upscale whichever texture happened to be bound first, and the ghosting would be reported as a renderer bug rather than as a wrong binding. upscale_bindings now resolves the dispatch's texture descriptors itself, the way build_descriptors resolves them -- same view type, same ViewUse::Sample, same handle_is_storage_pointer guard for the case Fritchii's d603d98bb fixed -- so index i is dw.imgs[i]. The seam sets colour_binding from the contract, and temporal_possible() can now be satisfied rather than refusing by construction. The per-network log reports the same list, so its roles are descriptor slots too. Its entries are now "slot:WxH=role" rather than the extent-and-format of whichever textures draw_sampled happened to hold, which is what makes the log usable for deciding what a backend reads. No behaviour change: nothing new is recorded into a command buffer, and the stand-in still takes the same dw.imgs[0] and dw.imgs[ntex] it always did. The resolve is a view lookup for handles the descriptor build has just resolved for this same dispatch. Tested with python -I -B tests/run_all.py: 89 passed, with the same 8 failures and 12 skips as upstream/main here (a missing clang++ or SPIR-V headers). The renderer is not compiled here -- a clone carries no CPM dependency set -- so this is reviewed against the descriptor build it mirrors, and the formats and extents it reports want checking against a real run before a backend trusts them. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first half of an FSR backend (docs/upscaling.md row 3): its shaders. Adds AMD's FidelityFX SDK v1.1.4 GPU sources under third_party/fidelityfx-sdk, unmodified and MIT, with a provenance note, and scripts/fsr3_shaders.py to compile the eleven variants into gpu/vk/shaders/fsr3_*_comp_spv.h. Eleven, not eight: the eight passes, plus wave-free variants of both pyramids for devices without the subgroup operations the single-pass downsampler prefers, and an accumulate compiled with sharpening on. The option set lives in one place in the script because each option is a promise about the inputs -- HDR colour, low-resolution motion that is not jittered, reversed depth -- and the host code has to hand the passes what they were compiled for. This is the SDK-source route rather than AMD's FidelityFX API, so no runtime library is needed, none is included, and one code path serves Windows and Linux. NotRexed#577 vendors AMD's API headers for the other route under third_party/fidelityfx; if both land, one should go, and the doc says so. The headers are checked in, like every other shader in gpu/vk/shaders, so a clone builds with no glslang. tests/test_fsr3_shaders.py recompiles all eleven and compares, and validates each module with spirv-val, so the headers cannot drift from their sources and an invalid module is caught here rather than on a device. It is wired into run_all's tool table, and glslang-tools is added to the Linux CI step that had spirv-tools only. That costs about 1.3 MB of generated hex. Generating at build time would remove it at the price of a glslang dependency for every build; the doc records the trade rather than deciding it. No renderer change: nothing includes these headers yet. The host side that records the passes is the next commit. Checked here: all eleven compile and pass spirv-val, the check passes, and it fails as it should on a changed word in a header, a changed compile option and a missing header. python -I -B tests/run_all.py: 90 passed, with the same 8 failures and 12 skips as upstream/main (a missing clang++ or SPIR-V headers). Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Whether a denoising upscaler can ever work here -- NVIDIA's Ray Reconstruction or AMD's Ray Regeneration -- turns on one question: does this game write a G-buffer. Neither takes colour, motion and depth; both want a ray-traced renderer's guides, which means albedo, normals, roughness and specular hit distance. I have been answering "no" from one piece of evidence, that nothing in gpu/ names such a target, which is weak: the replay does not name what it replays. A deferred renderer writes its guides as several colour attachments of one extent in one pass, so that is what to look for. RX_VK_MRT_CENSUS=1 logs each distinct render target set once -- the colour attachment count, their formats and the extent -- from targets(), which resolves once per NOP_SET_RT rather than per draw, so the cost is one line per distinct set and nothing at all while off. Capped at 512 distinct sets so a pathological run cannot fill a log. Never more than one colour attachment means there is no G-buffer and the feature is settled as not applicable to this title. Three or four at one extent means guides may exist and identifying them becomes ordinary work. Either way the docs can then state a finding instead of an assumption. docs/upscaling.md also records what is actually deliverable per vendor, and splits FSR 3 and FSR 4 into two backends rather than two versions: FSR 4 is not source-available, ships as prebuilt signed libraries in AMD's SDK releases, and is reached by loading AMD's runtime and asking for a provider, on AMD hardware. The API it is reached through is the FSR 3.1 API, so row 3 is the path to it. No behaviour change while the switch is unset. Tested with python -I -B tests/run_all.py: 90 passed, with the same 8 failures and 12 skips as upstream/main here. The census needs a run to produce its answer, which needs a built game. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…le it A 240 s run of the opening with RX_VK_MRT_CENSUS=1: 136 distinct target sets, 491 presents. The test I set was "if the game never binds more than one colour attachment, there is no G-buffer" -- and it binds more. Seven sets bind six, two bind three, six bind two. Reading them settles it the other way. The six-attachment sets are 1x1 through 64x64 with every attachment B10G11R11: six cubemap faces down a mip chain, an environment or irradiance probe. The two- and three-attachment sets are 170x96 to 768x432 in single-channel float pairs or HDR pairs, which is post-processing at a quarter resolution or less. The main scene is one HDR target at 2048x1152, and the present is 2560x1440 RGBA8 sRGB. A deferred G-buffer is three or four attachments at the main render resolution carrying albedo, packed normals and roughness. Nothing here is that, so there are still no guides for a denoising upscaler and Ray Reconstruction and Ray Regeneration stay inapplicable here. The conclusion is the one I had been asserting; the argument I was asserting it from (no target in gpu/ is named "normal") was not a sound one, and the census is. Also confirms the rest of the series runs: the per-network log prints its roles and verdict for real networks (the movie upscalers report "no upscaler inputs", correctly -- they sample one 1920x1080 texture and nothing else), and the census costs one line per distinct set. The gameplay TAAU dispatch was not reached in this run, so the seam has not fired yet: with RX_VK_PRECOMPILE=0 the opening runs at about 2 fps because every pipeline compiles on demand, which is not an fps measurement and does not get past the titles. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first half of the backend: what an evaluation needs before one can be recorded. gpu/vk/vk_fsr3.cpp builds the eleven pass pipelines from the SPIR-V the previous commit checked in, the two samplers every pass declares at bindings 1000 and 1001, and the SDK's eighteen internal images. The resource table and the per-pass binding lists are written out in the SDK's order rather than derived, because they are the SDK's and not ours: ffx_fsr3upscaler.cpp fixes the formats and sizes, and each pass file numbers its own bindings. One wrong entry is a validation error at best and a wrong picture at worst. The one place that knowingly departs from the SDK is the luma history, allocated R8G8B8A8_UNORM because that is what the pass files declare as its storage format, where the SDK allocates RGBA16F -- a storage format qualifier has to match. supported() asks once whether the device has push descriptors and every storage format the images need, so a device that cannot run the passes says so before an allocation fails mid-frame rather than after. prepare() re-makes the images when the render or upscale size changes and tears everything down rather than leaving a half-built set behind. Nothing calls this yet, so there is no behaviour change. The dispatch half needs one input it cannot get from the renderer alone: device_to_view_depth and tan_half_fov come from the game's near plane and field of view, which for this title means reading its camera out of the dispatch's own constant buffer. That is the next commit, and the seam logs what it would need. Builds clean with RX_VK_ONLY=1, 0 warnings. Linux, RTX 5060 Ti, driver 615.71.09. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The other half: the constants, the clears, the barriers and the nine dispatches, following the SDK's ffx_fsr3upscaler.cpp rather than inventing an order. cbFSR3Upscaler is laid out field for field with a static_assert on its 148 bytes, because every pass reads it by offset and a field out of place is a wrong picture rather than a compile error. What is faithful to the SDK and worth checking in review: the jitter phase count moves one step a frame toward 8*(ratio^2) instead of jumping; the SPD counter, its mips and the previous-nearest-depth scratch are cleared every frame while the accumulation and frame info are cleared only on a reset; the Lanczos weights are built once and copied in; the two ping-pong sets swap on parity at the end; and RCAS only runs when sharpening is asked for, otherwise the sharpening accumulate variant writes the output directly. Three things specific to this renderer rather than the reference. Barriers are synchronization2, which is what this tree loads (vkCmdPipelineBarrier is not in vk_loader.h). The constant blocks live in one host-visible mapped buffer with a set of slots per frame in flight, so writing one frame cannot disturb a frame still being read. And descriptors are pushed per pass, with the two samplers at the bindings the SDK's GLSL fixes them to. upscale() refuses rather than guesses. No colour, depth or motion is a refusal, and so is a missing near plane or field of view: device_to_view_depth and tan_half_fov are built from them, and a guessed camera makes disocclusion wrong in a way that reads as a renderer bug. Nothing calls this yet, so there is no behaviour change; wiring it to the seam needs the camera read out of the dispatch's own constant buffer, which is the next commit. Untested beyond compiling: builds clean with RX_VK_ONLY=1, 0 warnings, Linux, RTX 5060 Ti, driver 615.71.09. No pass has executed, so nothing here is claimed to produce a correct image yet. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…scaler
A warmed run reached gameplay and the per-network log answered the question
this series was built on:
vk: network fd029a317a9a4b94: dispatch 32x36x1, 1 textures 1 images:
0:1536x864; no upscaler inputs
One sampled texture, one written image, nothing else. No motion, no depth. All
eleven networks in the game are that shape: at most two sampled textures, and
never a motion or depth target among them.
So OoT's upscaler is a single-image neural upscaler, not a motion-vector
temporal one, and docs/upscaling.md's premise -- that a vendor upscaler takes
that dispatch's place -- was wrong. FSR 3.1 and DLSS Super Resolution are
temporal. They need motion and depth, and this dispatch is never handed either,
so temporal_possible() refuses at it and will keep refusing however a backend
is wired. The doc now says so above the section it contradicts.
The gate doing exactly that is the one good outcome: it refused rather than
handing FSR whichever texture happened to be bound, which is the failure the
descriptor-order commit was about.
What the run does confirm is that the opportunity is real. The input is
1536x864 against a 2048x1152 scene, so the game already renders the 3D at 75%
and upscales it. The inputs for a temporal upscaler have to be assembled
instead of taken: depth exists on the scene passes and has to be carried to the
resolve, and motion exists nowhere as a target and has to be derived from the
camera, which ghosts on anything moving by itself. That is NotRexed#577's integration
point, and this is the argument for it.
Also in here, since they were written before the run: FSR is wired to the seam
behind RX_VK_FSR=1 (it cannot fire, by the above, but the path is complete and
compiles), and the camera is no longer a precondition. The reference
implementation documents that device_to_view_depth only scales FSR's
disocclusion thresholds and uses a typical near plane and field of view, so
refusing without the real camera was stricter than the evidence warrants.
Builds clean with RX_VK_ONLY=1, 0 warnings.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ze 3D at 3840x2160, and the Resolution row reaching the 3D - Output resolution (host/nn_applet.c, nx2/oot_patches.py apply_output_size): the game's whole frame at 2048x1152 (its own), 2560x1440 or 3840x2160, docked, after a restart (HOS_OUTPUT_RES over it). The display size, the docked size list, the 3D's largest size and the scene's size follow it, and the graphics scale reads x1 for a chosen output; the widescreen picture is made at its height (host/hos_main.c). - 3840x2160 3D at the 4K output: the game's render targets' GPU pool (305 MB, for 2560x1440) ran out and the next target came back null. It grows with the output's pixels (687 MB at 4K; HOS_RT_POOL_MB over it), with the "Graphics" module heap that holds it, that heap's budget check counting its shipped use, and the leftover heap made from it kept at its shipped size. At 2048x1152 and 2560x1440 nothing changes. - Resolution: a fixed scale reaches the 3D (it kept the dynamic path's last one), sent in 1/10000 as the twelfth of the output the game draws at; the row shows the 3D's size for the chosen output, and follows the Output resolution row at once. - Test switches: HOS_RES_LOG (the 3D's scale in the log), RX_VK_SHOT_W/H (the screenshots' size), RX_VK_MENU_SCRIPT (timed menu keys) and RX_VK_MENU_DUMP (each drawing of the menu as a BMP).
…A's DLLs not included) - gpu/vk/vk_dlss.cpp, the DLSS part of vk_draw.cpp, vk/shaders/dlss_*.comp: the gameplay upscale network's place in the frame taken by DLSS, which draws the 3D at the DLSS mode's size and scales it to the output with the game's depth and its camera's motion. Ported from zortos293/Lichii (its Streamline loader, camera motion and their shaders; Lichii credits Duck912's Eglantine work). - NVIDIA Streamline v2.14.16's headers in third_party/streamline (MIT, gpu/PROVENANCE.md). sl.interposer.dll, sl.common.dll, sl.dlss.dll and nvngx_dlss.dll are loaded from a streamline folder beside the game exe, or the NVIDIA app's DLSS; the interposer and the model must carry a valid NVIDIA Corporation signature (WinVerifyTrust). Windows only: elsewhere DLSS is unavailable. - Menu rows DLSS (after a restart), DLSS mode (Ultra Performance .. DLAA, each with its size for the chosen output; it sets the 3D's scale in place of Resolution) and DLSS preset (J..M, at once); RX_DLSS, RX_DLSS_PRESET, RX_DLSS_DIR over them, RX_DLSS_DEBUG for its inputs. The scale sent to the game is for the output it runs at (the window crop). - readme: the rows, where NVIDIA's files come from; the Lichii credit.
The previous commit left the camera as a typical value, which is what the
reference implementation settles for because DLSS hands it none. It does not
have to be a guess here: sead's perspective projection carries it, and its
matrix update is already a patch site (nx2/oot_wide.py, main+0x3485cc0, the
whole function f_834a5cc0).
Reading that function's lifted C gives the layout, each offset from what the
code does with it:
+0x98, +0x9C the near and far planes, loaded as a pair and swapped by the
byte at +0x0A before their difference is taken, so that byte is
a reversed-depth flag
+0xAC tan(fovy / 2): 1.0f / it becomes m11, which is 1 / tan(fovy/2),
so the field of view is 2 * atan(it)
+0xB0 the aspect ratio, divided into m00 (and where the ultrawide
patch stores its own)
+0xB4, +0xB8 the projection's offsets, each doubled into m02 and m12
x0 is the projection object throughout that function and x1 is the 4x4 it
writes, so one injected block there can publish the near plane, the far plane
and the field of view to host globals the renderer reads, the way
hos_wide_ratio already travels the other way. It is also the same data
camera-derived motion vectors need, which is what the integration point this
series is moving to requires.
Docs only.
Assisted-by: Claude Opus 5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the stand-in Two fixes that the first run found. **The camera is no longer a typical value.** nx2/oot_camera.py appends a block to sead's perspective projection update (main+0x3485cc0, f_834a5cc0), where x0 is the projection object, that publishes the near plane, the far plane and tan(fovy/2) to host/hos_oot_camera.c. The renderer reads them and gives FSR the real camera; without them the Inputs defaults stand, which only scales FSR's disocclusion thresholds. docs/upscaling.md has how each offset was identified from what the function does with it. The site is shared with oot_wide.py's camera-ratio patch, so this appends to lifter.after rather than replacing it, and lift.py installs the ratio first. Which of +0x98 and +0x9C is the near plane depends on the byte at +0x0A that the function uses to swap them, so rather than decide what that flag means this takes the smaller as the near plane and the larger as the far, and publishes nothing unless both are positive and ordered. **A backend that is not the stand-in has to be reachable without it.** My previous commit put the FSR path inside taau_stand_in, after a guard that returns unless taau_key matches -- and on a card with hardware f16 matrix multiply taau_key defaults to "-", because the real network runs there. So FSR was unreachable on exactly the hardware that can run it. That is the same flaw the doc correction describes, reproduced in code one commit later. RX_VK_FSR=1 now brings the function in on its own, keyed by a list of its own (RX_VK_UPSCALE_KEYS, default the gameplay network) rather than by the stand-in's conditional default, and the stand-in only runs when it was asked for. Builds clean with RX_VK_ONLY=1, 0 warnings. The camera publish needs a full game build to take effect, since it is lifted C and a host global. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two inputs the game never hands its upscaler, assembled instead of taken. **The depth.** targets() now keeps the depth target bound with a colour target, the largest one seen, so a shadow map or a quarter-resolution effect pass does not displace the one the scene was drawn with. That is the only depth there is: the gameplay network samples one colour texture and nothing else. **The motion.** gpu/vk/shaders/camera_motion.comp reconstructs each pixel's world position from the reversed device depth with the inverse of this frame's view projection, projects it with the previous frame's, and writes where it was in render pixels, which is what FSR 3.1 is compiled for (LOW_RESOLUTION_MOTION_VECTORS=1, unjittered). Far-plane pixels, a degenerate reconstruction and anything behind the previous camera all get zero motion rather than a vector from nonsense. This is camera-only by construction. It is right for a world that does not move, and a character, foliage or a particle ghosts, because its own motion is not in the depth buffer. That belongs in whatever setting turns it on, not only in a comment. The host side makes the pipeline and an R16G16F target, inverts the matrix in doubles and refuses a singular one, and stands this frame's matrix in for the previous one on the first frame and after a reset, which makes the motion exactly zero rather than arbitrary. **What is still missing is the matrix.** sead's projection update gives the projection alone; a temporal upscaler reprojects with view * projection. Rather than guess which offset of which object holds the view, nx2/oot_camera.py now also publishes the guest addresses of the projection object and of the 4x4 it writes, and HOS_CAMERA_DUMP=1 logs a window of floats around both so the matrix can be identified from what it contains. Guest addresses only. The seam's log also reports the camera it was given and the scene depth it found, so one run says whether both arrived. Builds clean with RX_VK_ONLY=1, 0 warnings; the shader compiles and passes spirv-val. Nothing calls the motion pass yet. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured before any of this changes anything, because it decides whether an upscaler is worth having here. Linux, RTX 5060 Ti, driver 615.71.09, headless under Xvfb with a warmed shader cache, the opening and Kokiri Forest: 57-60 fps, held over 33,600 presents in one run and 9,000 in another 1106 draws (0 skipped), 99 dispatches, 148 passes a frame 1536x864 internal, resolved to a 2048x1152 scene, presented at 2560x1440 Ocarina already runs at its target frame rate. An upscaler is not what gets it there, and the argument for FSR or DLSS in this title is image quality at a given cost, or headroom in areas this run did not reach -- not frames. Anyone reading this document to decide whether to spend time on it should see that first. 1106 draws and 148 passes a frame at 60 fps is also not an upscaler-shaped problem, which is worth saying where someone is choosing what to work on. Docs only. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The baseline section said Ocarina already ran at its target frame rate, which misstates the objective. 60 is the floor. The limiter at mod + 0xfc takes any rate from 40 up when variable refresh is on, so HOS_FPS_TARGET can ask for more, and the real-time frame step at main+0x6e6cc4 keeps game speed honest at whatever comes out. What an upscaler buys against a floor is the two things a floor does not give: a higher internal resolution at the same rate, and the headroom to hold a higher rate at all. The scene is 1536x864 for a 2560x1440 present, so there is a lot of resolution being left behind, and that is the gap FSR or DLSS closes. Docs only. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The seam now builds the inputs rather than looking for them in a dispatch that does not carry them: the scene depth targets() kept, and motion reprojected from it by the camera-motion pass. Only where the depth's extent matches the network's input, so a shadow map or an effect pass cannot stand in for the buffer the scene was drawn with. The Request is rebuilt for the assembled set before temporal_possible() sees it, because the gate is about what a backend will be handed and that is no longer the dispatch's own bindings. Motion scale is 1 because the pass already writes render pixels. It stays off for one honest reason, which is the last one: hos_oot_view_projection answers 0. sead's projection update gives the projection alone and the view is not identified yet, so reprojecting would be through a matrix that is not the one the depth came from. HOS_CAMERA_DUMP=1 is how that gets found; until it is, camera motion does not run and FSR falls through, rather than producing a plausible picture built on the wrong matrix. Builds clean with RX_VK_ONLY=1, 0 warnings. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run reached gameplay and published nothing. The reason is in oot_wide.py's own comment: sead's perspective projection matrix update has "three inlined copies", and the path that actually runs is one of them, not the out-of-line function I patched. So the publish now goes at all four sites oot_wide patches for the camera ratio, with the projection object taken from the register each copy holds it in -- x0, x21, x0 and x2, which are the second argument of oot_wide's _ratio() at the same offsets. The 4x4's address is published only from the out-of-line copy, where x1 was verified to hold it by reading the function. The dump dereferences that address, so publishing a guessed one from an inlined copy could fault rather than merely log nonsense. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run with HOS_FPS_TARGET=120 presents a locked 60 and logs "frame limiter 60" at every sample, 300 presents apart. The log reads +0xfc before the raise, so reading 60 every time means the engine writes it back between samples: with variable refresh on, the adaptive controller owns that word, exactly as the comment above it says. HOS_FPS_FORCE=1 also raises the three controller words that sample reads as 60 (+0x270+0x58, +0x28, +0x2c) and the nominal rate at +0x2c8, to find out whether the rate the controller settles on is what caps the frame or whether something else does. Labelled an experiment and off by default on purpose: it writes engine fields every frame whose meaning is inferred from one log line, and the real-time frame step already derives game speed from +0x2c8, so forcing that word changes two things at once. If it turns out to be the cap, the fix belongs at whatever writes the controller rather than in a host loop fighting it. Assisted-by: Claude Opus 5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…on top of NotRexed#850's DLSS frame code) A second backend for NotRexed#850's replacement of OoT's gameplay upscale, ported from the FSR 4.1 work on Lichii's OoT branch (madebyjordan/Lichii-Hyrule oot-dev, 7f8ae5b337..c5f2fce17d). It takes the same colour, depth and camera motion as DLSS; DLSS keeps priority when it is on. - vk_fsr4.cpp: AMD's FSR API (FidelityFX SDK 2.4). Only the MIT headers are vendored (third_party/fidelityfx); AMD's signed loader and upscaler DLLs load at run time from RX_FSR4_DIR or beside hookshot.exe. The newest 4.x provider the loader lists is picked explicitly (RDNA 2 and 3 run its INT8 model). LDR, non-linear colour, standard depth. One context per output size, render sizes up to it, so the Resolution row and the game's dynamic resolution need no new context. - Jitter: the game draws without one, so the draws that test or write the scene's depth move by a Halton (2, 3) sub-pixel offset through the viewport, and FSR is told it. The offset is part of the state dyn_pack compares, or a frame whose draws match the last frame's would keep the old one. From Lichii: the phase moves on at the network's pass (where the game's frame ends), not at presents; it is on only while FSR drew the last network pass. - AMD edge line (Lichii, RX 6700 XT): moved up or left, the AMD driver draws nothing into the scene's last column or row, a stale near-white value that became a flickering line in FSR's history. FSR's colour and the motion's depth repeat the column and row before them. Colour going into FSR is cleaned of NaN and Inf and clamped to [0, 1]; the output pass drops NaN and Inf for both upscalers. - FSR history resets when FSR did not draw the last network pass or on a camera cut, not by present count. - Menu: "AMD FSR 4.1" (Off/On, at once; ini key fsr4, RX_FSR4=0/1). Resolution sets what it upscales from (75% Quality, 50% Performance, 100% anti-aliasing only). RX_FSR4_JITTER=0 turns the jitter off. - What changes on native, compared with Lichii: no GPU-time divisor (Hookshot already scales the game's timer, so it keeps full quality), no hunt for the tonemap draw or depth reader (NotRexed#818 takes them where the draws are), no 4/3-only size check. - tests/test_fsr_jitter.py: the jitter is compared and applied, moves on at the network's pass, and the edge repeat is in both passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… images given back - The FSR context is made with FFX_UPSCALE_ENABLE_DYNAMIC_RESOLUTION: the render size changes under it with the Resolution row and the game's dynamic resolution. - vk_fsr4.cpp is Windows only (AMD's signed runtime is), with stubs elsewhere: the Linux build has no LoadLibraryExA. - AMD's dispatches count as work (note_work, vk_work.h), so full_barrier does not leave out the barrier after them as a repeat. - dlss_image gives a replaced image back 120 presents later, as typed_upscale does with its histories, instead of keeping it: DLSS fixes the render size, but FSR can run with Resolution Auto. - The dlss_*.comp headers regenerated at SPIR-V 1.6, as NotRexed#850 makes them. - readme: FSR 4 claims limited to what was tested (RX 6700 XT, INT8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There is no AMD loader on Linux, so the upscaler library itself (amd_fidelityfx_upscaler_vk.so) is loaded with dlopen from RX_FSR4_DIR or beside the game. A Vulkan FSR 4 build reads the SDK's older backend description and records with descriptor buffers: the description is chained next to the current one, and VK_EXT_descriptor_buffer is enabled only when the library is there. Windows is unchanged. BITMAPFILEHEADER joins the POSIX windows.h shim, for the menu dump. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… exposure A Vulkan FSR 4 build lists its Auto provider before INT8 and FP8, so taking the last of equal versions chose FP8 on hardware without it. It also rejects a dispatch without an exposure: the game's colour is already tonemapped, so it gets a 1x1 R32F of 1. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The file was Windows-only, which hid the Linux loading. Remove the guard and its stubs, and say in the readme what Linux needs. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring the FSR 3.1 compute passes, the upscale seam and the input contract onto the FSR 4 Linux branch. The only conflict is the network dispatch log in vk_draw.cpp: keep the signature that logs again on a size change and the frame number, and add the contract's roles to the same line. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The renderer hands FSR tonemapped, square-root encoded colour (LDR), a standard D32 depth and render-pixel motion, so build the passes that way (HDR input off, inverted depth off) instead of converting the inputs. Read the camera for finite standard depth, bind the SDK's default exposure (an image of zero reads as 1.0), and use the no-wave pyramids where the device lacks the subgroup operations. The images of an old size were destroyed while in flight and the new ones never left UNDEFINED: retire them and restart the history instead. Add available() and reason() so the menu can show why it cannot run. Regenerate the headers (glslang's version comment ignored by --check, same SPIR-V) and pin the options in tests/test_fsr3_shaders.py. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the AMD FSR 4.1 row with an FSR upscaler row: Off, Auto, FSR 4.1 and FSR 3.1, changed at once. Auto takes FSR 4.1 when AMD's library is present with a model for the card, else the built-in FSR 3.1, else the game's upscale. The row shows the engine in use and why a forced choice cannot run; FSR 4.1 failing in play falls back to FSR 3.1. FSR 3.1 runs on the chain FSR 4.1 uses, from the same colour, depth, motion and jitter, with near, far and field of view read from the camera matrix; the output goes through dlss_output.comp as before, and the history restarts whenever the engine changes. RX_FSR4=0/1 and an old ini's fsr4= still work (Off and Auto); RX_UPSCALER and upscaler= are the new keys. The camera symbols of the host are weak, so this renderer also loads under a game built without them. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
N3oTraX
force-pushed
the
linux/fsr3-fallback
branch
from
November 9, 2026 00:04
1c53917 to
6095c4d
Compare
N3oTraX
marked this pull request as ready for review
November 9, 2026 00:06
This was referenced Nov 9, 2026
Author
This was referenced Nov 9, 2026
Author
|
@NotRexed Done ! On top of 2.0.7. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One Insert menu row chooses the upscaler: Off, Auto, FSR 4.1, FSR 3.1, changed at once while playing. FSR 3.1 needs no external library, so cards and systems without FSR 4 still get a temporal upscaler.
What changed
HDR_COLOR_INPUT=0,INVERTED_DEPTH=0, low-resolution motion, not jittered. Near, far and field of view come from the camera used for the motion.tests/test_fsr3_shaders.pypins the options.AMD FSR 4.1row becomesFSR upscaler. Auto takes FSR 4.1 when the library is present with a model for the card, else FSR 3.1, else the game's upscale. The row shows the engine in use, and why a forced choice cannot run. If FSR 4.1 fails in play, FSR 3.1 takes over.RX_FSR4=0/1andfsr4=still read as Off and Auto. New:RX_UPSCALER=off|auto|fsr4|fsr3andupscaler=.UNDEFINED;hos_oot_cameraandhos_oot_view_projectionare weak, so the renderer still loads with a host built without them.One conflict with #822, in the network dispatch log in
vk_draw.cpp: both behaviours are kept.How it was tested
e0f3ccfa, Ultrawide on5120x2160 -> 5120x2160, switched live several times from the menu. FSR 3.1 reportedcamera near 0.100 far 25040.7, and all 11 passes were created. No crash or device lost in the logtests/test_fsr3_shaders.py: 11 modules valid withspirv-val.tests/run_all.py fsr menu upscale resolution: 7 passed, 1 skipped (test_upscale_contract, no Vulkan headers found; passes by hand). Full suite: 98 passed, 13 skipped, 1 failed (test_mem_trim, which fails the same way on the commit below this work)Image quality was not compared and fps was not measured.
Run again after the rebase (
6095c4d, full game rebuilt): same GPU and driver, FSR 4.1.1 INT8 and FSR 3.1 each drew5120x2160 -> 5120x2160, switched live several times from the menu, no crash or device lost in the log.The run used
RX_VK_NO_WARM=1,RX_VK_NO_PREDICT=1andRX_VK_PRECOMPILE=0because of #803, andRX_VK_ANISO=16.Not tested
RX_VK_FSR=1route now runs on the same LDR shaders and was not revalidated.🤖 Generated with Claude Code