Skip to content

feat(vk): FXAA in the gameplay TAAU stand-in (edges on cards without the network) - #1115

Merged
hckrman101 merged 1 commit into
NotRexed:mainfrom
t0masz407:feat/taau-fxaa
Nov 9, 2026
Merged

hckrman101 merged 1 commit into
NotRexed:mainfrom
t0masz407:feat/taau-fxaa

Conversation

@t0masz407

Copy link
Copy Markdown
Contributor

What

The gameplay TAAU stand-in (taau_stand_in → typed_upscale, shaders/taau_upscale.comp) now runs FXAA on its input instead of the plain linear sample. It uses FXAA 3.11's quality variant, in the network's perceptual (square-root) encoding, at the input's texel size.

  • RX_VK_TAAU_FXAA=0: the linear upscale as before.
  • RX_VK_TAAU_FXAA=2: diagnostic colours for the pixels it moves, red along an edge and blue for the sub-pixel blend.

The other typed_upscale caller (the generic RGB9E5 fallback) is unchanged.

Why

On cards where the TAAU network does not run (software matrix multiplies: AMD, Intel; #52, tracked in #1006), the picture has no anti-aliasing. Every edge stays stair-stepped, and hair, Link's cap, leaves and thin branches shimmer as they move. The network reads the scene colour alone: its dispatch binds one texture (network fd029a317a9a4b94: … 1 textures 1 images), with no motion vectors or depth. Temporal accumulation would therefore smear everything that moves (the history bound's comment records that). A one-frame filter is what can stand in.

Tuning

At FXAA's default thresholds (0.125 / 0.0312, sub-pixel 0.75), RX_VK_TAAU_FXAA=2 showed most of Kokiri Forest in red: moss, cloth and leaf textures were taken for edges and softened. At the high-quality thresholds (0.166 / 0.0833) with half the sub-pixel blend (0.5), it acts on contours: Link against the background, the hair's outline, the cap, leaf edges. Textures keep their plain sample.

The push constant's third word is now read as bits: 1 the history bound (RX_VK_TAAU_CLAMP), 2 FXAA, 4 its colours. The bound tests its own bit only.

Tested

RX 7900 XTX, RADV (Mesa 26.2.6), Linux (Hyprland, native Wayland), v2.0.7 plus this change, Kokiri Forest at 3840x2160 output with dynamic resolution (the 3D at 2880x1620):

  • edges of hair, cap and foliage visibly smoother, and textures not blurred, compared side by side with RX_VK_TAAU_FXAA=0;
  • RX_VK_TAAU_FXAA=2 before and after the threshold change, as described above;
  • no frame-rate change noticed in play (84-89 fps shown, 41-44 the game's own). This is not a controlled measurement; the filter reads 5 texels per output pixel off edges and up to about 30 on one.

tests/test_taau_fxaa.py checks:

  • the history bound stays on its own bit;
  • only the gameplay stand-in sets FXAA's bits;
  • RX_VK_TAAU_FXAA switches it;
  • with glslangValidator of the header's version (16.6.0) on PATH, taau_upscale_comp_spv.h is compiled again from taau_upscale.comp and compared.

🤖 Generated with Claude Code

…the network)

Where the TAAU network does not run (software matrix multiplies: AMD, Intel), its stand-in
upscaled the scene colour linearly, and every edge stayed stair-stepped: hair, the cap, leaves
and thin branches shimmer as they move. The network reads the scene colour alone (no motion
vectors, no depth), so a one-frame filter stands in: FXAA 3.11's quality variant on the input,
in its perceptual encoding, at FXAA's high-quality thresholds and half its sub-pixel blend (at
the default ones it also took moss, cloth and leaf textures for edges).

RX_VK_TAAU_FXAA=0: the plain linear upscale as before; =2: the pixels it moves drawn red (along
an edge) or blue (sub-pixel blend). tests/test_taau_fxaa.py keeps the history bound on its own
bit, FXAA to the gameplay stand-in, and the SPIR-V header compiled from its source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013mj4MdWTJo5ht4ShZX8Vpp

@hckrman101 hckrman101 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, small and well contained. FXAA sits behind its own push-constant bit inside the existing taau_upscale.comp dispatch, so there are no new images, descriptors, barriers or layout changes. Bindings, the push-constant range (still 12 bytes) and the history and resize handling all stay as they were. Only taau_stand_in's temporal=true call sets bits 2 and 4. The generic RGB9E5 fallback (temporal=false) is untouched, and the history bound now tests & 1, so FXAA can't turn the clamp on by accident.

What I checked:

  • The port follows FXAA 3.11 quality step by step. That covers the N/S to W/E swap after ns/we are taken, pair_n and the sign flip, the half-texel posB, end-of-span search with grad/4, good span, and the sub-pixel term over 12 with max(span, sub). The search schedule is a custom 12-step one, which is fine.
  • Luma is taken in the sqrt-encoded input, which is the perceptual space FXAA expects. The square is applied afterwards, as before.
  • taau_upscale_comp_spv.h matches the source. I rebuilt it with glslangValidator 16.6.0 (-V --target-env vulkan1.3 --vn TAAU_UPSCALE_COMP_SPV) and got a byte-identical file, and spirv-val passes. Of the -107 lines, 104 are the regenerated header and 3 are shader lines replaced by the bit tests.
  • DLSS/FSR: on #1008 (and the #841 chain it sits on), temporal_stand_in/dlss_stand_in runs before taau_stand_in, and #1008's temporal_rejected_network goes through upscale_stub, not typed_upscale. On #1335, FSR 3/4 are chosen inside taau_stand_in before it falls back to typed_upscale. So FXAA only runs when no temporal backend took the dispatch. The size[2] context lines are the same on both branches, so I don't expect a textual conflict in vk_draw.cpp, and neither branch touches taau_upscale.comp.
  • No settings option is added (env var only), so no option-id collisions. tests/test_taau_fxaa.py is picked up automatically by run_all.py's tests/test_*.py glob.

Nits only, inline. One design note, no change needed: the 3x3 neighbourhood is read at the output pixel's centre offset by input texels, so at a 1.33x scale every tap is already a bilinear blend. That's a reasonable choice, since snapping to input texel centres would go blocky, but it explains why the edge detection is a bit softer than native-res FXAA. FXAA is also on by default for everyone on the stand-in path with no menu toggle. That's fine as a default, and it could later go in #1335's Upscaling row if people want it off without an env var.

No CI has run on this branch yet (fork workflows not approved), so the test hasn't run in CI.

const float range = hi - lo;
// (FXAA's high-quality thresholds: with 0.125 / 0.0312 it also took the moss, cloth and leaf textures for edges and
// softened them)
if (range < max(0.0833, hi * 0.166)) return rgbM; // (no edge: flat areas and fine textures keep their plain sample)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: an inf texel (the header mentions frames that blow up) gives range = inf and hi * 0.166 = inf, so this test is false and FXAA carries on. Further down, sub becomes inf/inf = NaN, and clamp/max with NaN are undefined in SPIR-V, so the final uv can come out NaN for the bad pixel and its neighbours. That's harmless on RADV/NVIDIA in practice, since their fmax drops the NaN. Still, something like if (!(range < 65504.0)) return rgbM; (or an isinf/isnan check on hi) would keep this path from making a bad texel worse when the clamp is off, which is the default.

Comment thread gpu/vk/vk_draw.cpp
const uint64_t now_frame = texture_frame_no();
size[2] = temporal && !no_clamp && hist_frame && now_frame - hist_frame <= 3 ? 1u : 0u;
// (the gameplay stand-in's edges: FXAA in the shader. RX_VK_TAAU_FXAA=0: the plain linear upscale as before; 2: the
// pixels it changed drawn red)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: mode 2 draws the sub-pixel blend in blue as well as edges in red (shader line 83), so this comment could say "red/blue" to match the shader and the PR description.

@hckrman101
hckrman101 merged commit bc331a2 into NotRexed:main Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants