Integrate RES audio updates on the audio sigma schedule - #7
Open
morluto wants to merge 1 commit into
Open
Conversation
morluto
marked this pull request as ready for review
August 11, 2026 10:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6.
Problem
The RES denoiser currently expresses the audio update in the video-sigma
coordinate:
It then passes that estimate to
h3_res_stepwithsigmas.video.This is a valid continuous-time reparameterization, but it makes the denoised
forcing nonlinear even for constant audio velocity. At sparse step counts,
the RES discretization consequently has substantially more endpoint error
than the native audio-sigma formulation.
Change
This PR keeps each velocity conversion paired with the schedule used by RES:
flowchart LR VV["Video velocity"] --> VD["x + sigma_video * v"] VD --> VR["RES on sigmas.video"] AV["Audio velocity"] --> AD["x + sigma_audio * v"] AD --> AR["RES on sigmas.audio"]The shared velocity-step helper constructs the denoised estimate using the
same sigma grid passed to RES. The audio wrapper owns the
schedule.audioinvariant, preventing the coefficient and integration grid from drifting
apart again.
h3_dit_denoisenow applies video velocity onsigmas.videoand audiovelocity on
sigmas.audio. The Euler paths, schedule construction, and DiTforward pass are unchanged.
Regression evidence
For constant velocity,$D_a=x+\sigma_a v_a$ is constant, giving an exact
solver oracle. A scalar sample starting at
1.0with velocity0.5has theexact endpoint
1.5at sigma zero.The regression covers all three step counts and accepts an absolute endpoint
error below
1e-6.A mutation check restoring the previous coefficient and video grid fails at
four steps:
Restoring the native audio formulation passes all three cases. These numbers
are deterministic solver-state measurements; they do not represent decoded
audio quality percentages.
Validation
git diff --checkpasses.