Skip to content

Custom Shader

SBoys3 edited this page Jun 14, 2026 · 2 revisions

Custom Shader Documentation

The Custom Shader is a powerful feature that replaces SteamVR's default distortion shaders with enhanced versions that provide significant visual improvements for supported headsets. This documentation covers all available settings and their effects.

Important Notes

  • Disabled by Default: The custom shader is disabled by default because it has the potential to break with future versions of SteamVR
  • Settings Location: All custom shader settings are found in the General tab of the Custom Headset GUI under the Custom Shader section
  • Changes Apply Immediately: Settings take effect immediately without requiring a restart
  • Compatibility Warning: The custom shader has the potential to completely break SteamVR while enabled after a future SteamVR update. Disable it if you experience issues or compatibility problems.

Enable Custom Shader

Toggle: Enable/Disable the custom shader system

When enabled, this replaces SteamVR's default distortion shaders with the custom shader that provides all the features documented below.


Enable for Other Native HMDs

Toggle: Apply the custom shader to non-Custom Headset drivers (e.g., official Vive, Index, Beyond, etc.)

This allows you to use custom shader features like sampling filters and color correction with other native VR headsets. Non-native SteamVR headsets (like Quest, etc.) will not work.


Sampling Filter

The sampling filter are run directly in the distortion shader and therefore can server as upscaling filters. Different filters offer different trade-offs between sharpness, aliasing reduction, and performance.

Filter Comparison

I prefer disabling anti-aliasing in games and then using FXAA2 CAS. It filter does somewhat work like an upscaler despite being an amalgamation of an anti aliasing and sharpening function. It makes the image sharper while avoiding shimmering.

Different filters have different amounts of samples. Higher amounts of samples have more of a performance cost. The cost scales with the resolution of the headset display, not the SteamVR resolution.

Filter Texture Samples per Subpixel
None (Bilinear) 1
FXAA2 CAS 13
Contrast Adaptive Sharpen (CAS) 9
FXAA2 7
Luma Sharpen (Fast) 2
Luma Sharpen (Normal) 4
Luma Sharpen (Wider) 4
Luma Sharpen (Pyramid) 4
Nearest Neighbor 1

Filter Details

None (Bilinear) - Default

  • Description: Standard bilinear interpolation. Fast and smooth but can appear soft.

FXAA2 CAS (FXAA2 + Contrast Adaptive Sharpening)

  • Description: Combines FXAA2 with CAS for anti-aliasing and sharpening in one pass.
  • Additional Settings:
    • CAS Sharpening (0.0-2.0): Intensity of the CAS sharpening effect
    • CAS Contrast (0.0-1.0): How much the shader adapts to high-contrast areas

Contrast Adaptive Sharpen (CAS)

  • Description: Edge-aware sharpening that adapts to local contrast. Sharpens edges while preserving smooth areas.
  • Additional Settings:
    • Sharpening (0.1-2.0): Intensity of the sharpening effect
    • Contrast (0.0-1.0): How much the shader adapts to high-contrast areas

FXAA2 (Fast Approximate Anti-Aliasing)

  • Description: Uses FXAA as an upsampling method. Reduces jagged edges but can introduce some blur.
  • Additional Settings:
    • Sharpen Strength (0.0-5.0): Adds sharpening to counteract FXAA blur
    • Sharpen Clamp (0.0-0.3): Limits maximum sharpening to prevent halo artifacts

Luma Sharpen

  • Description: A cheap sharpening filter with multiple sampling patterns that avoids colored artifacts.
  • Additional Settings:
    • Strength (0.1-5.0): Intensity of the sharpening effect
    • Clamp (0.0-0.3): Limits maximum sharpening to prevent halo artifacts
    • Pattern:
      • Fast 7-tap Gaussian, fastest but least accurate
      • Normal 9-tap Gaussian, Standard luma sharpening filter
      • Wider 17-tap Gaussian, A wider filter
      • Pyramid 9-tap high-pass filter
    • Radius (0.1-6.0): Controls the sampling radius; higher values can sharpen blurrier source images.

Nearest Neighbor

  • Samples: 1 per subpixel
  • Description: No interpolation; each output pixel uses the nearest input pixel. Can appear blocky but preserves sharp edges and is fastest.

Enable Filter for Overlay

Toggle: Apply the sampling filter to UI overlays

When disabled (default), UI overlays like the SteamVR dashboard use simple bilinear sampling instead of the configured sampling filter. This improves performance by only applying complex filtering to the game. When enabled, the same sampling filter is applied to overlays as to the main scene.

Note: Even when the menu is closed and there are no overlays, enabling this causes some decrease in performance. The SteamVR background is also an overlay, so the filters will have no effect outside of games when this is off.

Enable Filter for Dashboard

Toggle: Apply the sampling filter when the SteamVR dashboard is open When enabled, the sampling filter activates for overlays only when the SteamVR dashboard is actually visible, so there is no performance impact when the dashboard is closed. This does cause a brief lag spike when opening or closing the dashboard. The driver cannot detect if the menu is open at startup, so the dashboard will need to be re-opened for this to take effect.


Contrast

Range: 1-100 (default: 50)

Adjusts the contrast of the image. Values above 50 increase contrast (darker darks, brighter brights), while values below 50 decrease contrast.

Contrast Midpoint

Range: 0-100 (default: 50)

Adjusts the midpoint of the contrast curve. Lower values make the image brighter, higher values make it darker.

Contrast Linear

Toggle: Apply contrast in linear color space (before gamma)

When enabled, contrast is applied before gamma correction, which can produce more natural results in some scenarios.

Per-Eye Contrast

Toggle: Enable independent contrast controls for left and right eyes

When enabled, you can adjust contrast separately for each eye to compensate for display unevenness. The same settings are available.


Saturation

Range: 0-100 (default: 50)

Adjusts the color saturation of the image:

  • 0: Grayscale (no color)
  • 50: Normal saturation
  • 100: High saturation (colors are exaggerated)
  • 1000: Deep fry (goes well with -200 contrast)

Gamma

Range: 0.1-4.0 (default: 2.2)

Adjusts the gamma curve of the image. Higher values make darker areas brighter, while lower values make dark areas darker. The default of 2.2 matches standard sRGB gamma.


Subpixel Shift

Toggle: Enable subpixel-level sampling for improved sharpness

This feature samples each color channel (R, G, B) from slightly different positions that correspond to the actual physical location of subpixels on the display. This can dramatically improve sharpness:

  • MeganeX and Dream Air: Can extract detail from up to 500% of 1:1 resolution due to the hexagonal subpixel arrangement
  • Vive: Also supported with slight improvements
  • Other headsets: Has no effect

Note: This feature has essentially zero performance cost.

Shadertoy Demo

A visual demonstration of the subpixel shift effect is available at: https://www.shadertoy.com/view/Wcd3D7


Disable Mura Correction

Toggle: Force-disable Mura panel uniformity correction

Some headsets like the Vive have built-in Mura correction to compensate for OLED panel uniformity issues. This setting forcibly disables that correction if you prefer the uncorrected image.


Disable Black Levels

Toggle: Disable the black level clamp

Some headsets apply a black level clamp that can make dark areas appear slightly grey. Disabling this allows true blacks but may increase smearing on some OLED displays.


sRGB Color Correction

Toggle: Map sRGB content to the correct portion of the display's color gamut

The MeganeX and Dream Air uses a DCI-P3 display which has a wider color gamut than standard sRGB. Without correction, colors appear oversaturated because sRGB content is stretched to fill the entire DCI-P3 range. Enabling this option maps sRGB colors to their correct position within DCI-P3, resulting in more accurate colors.

This has no effect on other headsets.

Also reduces banding at the edges of the color space on BOE panels in the MeganeX.

sRGB White Point Correction

Toggle: Correct the white point in addition to the color gamut mapping

When enabled, uses an alternative color correction matrix that also corrects the display's white point to the official DCI-P3 standard. This can result in a slightly warmer or cooler image depending on the display, but provides more accurate color reproduction overall.


Lens Color Correction

Toggle: Correct for color variations across the lens surface

Applies a radial color correction to compensate for color non-uniformity across the lens surface:

  • MeganeX: The center of the lenses has a warmer (more red/green) color cast. The correction adjusts the red, green, and blue channels based on distance from center to compensate.
  • Dream Air: Has a red tint around the edges of the lenses. The correction reduces the red channel and slightly adjusts green toward the periphery.

This results in more uniform colors across the entire field of view. Has no effect on other headsets.


10-Bit Dithering

Toggle: Enable temporal and spatial dithering for 10-bit color depth

This feature allows 10-bit color content to be displayed on 8-bit panels by dithering across both space (4 adjacent pixels) and time (4 frames). This reduces visible color banding in gradients.

Note: Only effective with VR applications that output 10-bit color and almost nothing does.


Color Tint

Color Picker: Adjust RGB color balance

Allows fine-tuning of the overall color balance. Can also be used to increase or decrease overall brightness by adjusting all channels equally using the V slider.


Technical Documentation

Overview

The custom shader system works by hooking into the ID3D11Device::CreatePixelShader function to intercept and replace SteamVR's default distortion shaders with custom-compiled versions. The shader source is written in HLSL (High-Level Shading Language) and is compiled at runtime. All settings are controlled with preprocessor defines and are changed by recompiling the shader with new settings. This means that disabled settings are completely removed from the shader.

Hot Reloading

The shader system watches the shader source directory for file changes. When the HLSL source file is modified, the shader automatically reloads without requiring a SteamVR restart. You can use this to develop or modify the shader.

Key Files

  • Shader Source: distort_ps_layered.hlsl - The main pixel shader source file
  • Shader Replacement Logic: ShaderReplacement.cpp - Handles shader hooking and replacement

How It Works

  1. Shader Hooking: When the driver initializes, it hooks the CreatePixelShader function of the Direct3D 11 device
  2. Shader Identification: Each shader is identified by the first 32 bytes of its bytecode (which contains a hash)
  3. Shader Replacement: When a matching shader is found, it is replaced with a custom-compiled version
  4. Runtime Compilation: The custom shader is compiled from the HLSL source using D3DCompileFromFile with various preprocessor defines based on current settings
  5. Shader Cache: Shaders are cached based on a hash of the code and defines, this avoids the cost of re-compilation and decreases the length of the lag spike when reloading shaders.

Performance Considerations

  • Subpixel Shift: Zero computational cost - only UV coordinate adjustments
  • Color Correction: Minimal cost - single matrix multiplication
  • Sampling Filters: Varies by filter - more samples = higher GPU memory bandwidth usage
  • Dithering: Minimal cost - simple arithmetic operations

Debugging

  • Compiler error messages are saved to .txt files next to the shader source for inspection
  • The shader can be forced to reload by modifying the source file
  • vrcompositor.exe can be started in Visual Studio to get log output.