Clean up cross-color and cross-luma artifacts in old PAL/NTSC footage, for VapourSynth and AviSynth+.
If your source came off tape or a cheap decoder and has rainbow shimmer on fine detail (cross-color) or crawling dots along sharp edges (cross-luma / dot crawl), this plugin can take a lot of it back out. It re-encodes the picture to the composite signal the bad decoder would have seen, then decodes that signal properly with a comb / Transform separator — so the artifacts the original decoder baked in never get re-created.
You do not need to understand any of that to use it. Feed a clip to
Restore and get a cleaner clip back.
VapourSynth:
import vapoursynth as vs
core = vs.core
clip = core.bs.VideoSource("tape.mkv") # your NTSC/PAL source
out = core.composite.Restore(clip, standard="ntsc")
out.set_output()AviSynth+:
BSVideoSource("tape.mkv")
composite_Restore(standard="ntsc")
That is the whole thing. Everything below is tuning.
- Format: any constant-format YUV clip (
RestoreandEncoderesample internally, so 4:2:0, 4:2:2, 8-bit, 10-bit — all fine). - Width: any width. It is resampled to the composite raster and
back; pick your output width with
width=(default 720). - Height: this is the one hard rule, because it fixes where the
picture sits in the TV raster:
- PAL — 576 lines, exactly.
- NTSC — 480 or 486 lines. A 480-line clip is placed on the
486-line raster automatically from its field order (
_FieldBased): BFF/DV content at rows 4–483, TFF/RP 202 content at rows 5–484. - Anything else is an error, so resize/pad to 576 or 480/486 first.
Field order matters for NTSC 480-line input — set _FieldBased correctly
(most DV/tape captures are BFF) or the raster placement, and therefore
the subcarrier phase, will be wrong.
out = core.composite.Restore(clip, standard="ntsc")The whole round trip in one call: resample to the composite raster,
re-encode, decode with a good separator, resample back to width. This
is the function you want for cleanup. It takes every Decode parameter
(see below) plus two of its own:
refine— luma detail recovery, default1. A short Y-only loop that recovers luma detail the original decoder softened. Chroma is untouched.0disables it; higher values (2–4) push harder on clean, detailed sources. See Aboutrefinebelow — it is not a sharpener.precomb— passed to the internal encoder (seeEncode); leave it off for cleanup.
Most users only ever set standard and maybe width. Reach for the
Decode parameters below only if the defaults leave something on the
table.
A sharpener guesses: it finds an edge and adds contrast around it,
inventing high-frequency content that was never in the signal. refine
does the opposite — it solves for the detail that the bad decoder
attenuated, using the fact that we know exactly how that decoder blurs.
The plugin already has a model of the crude decoder (it's what the round
trip is built on). refine runs a few steps of a constrained
deconvolution (a Y-only Landweber iteration): it proposes a sharper
luma, pushes it back through the modeled bad decoder, and checks
whether the result matches the luma you actually captured. It keeps only
the correction that makes the model reproduce your real footage. Nothing
is added that isn't required to explain the picture the decoder
produced — so the recovered detail is inferred from the signal and the
known blur, not painted on.
We measured this rather than assuming it. On real detailed footage the
extra high-frequency energy refine=1 produces moves the image closer
to a clean reference (higher SSIMULACRA2, luma PSNR, and XPSNR), and it
survives a downstream sharpener — the signature of genuine detail, not
invented edges. It even reduces the error in flat regions there, the
opposite of grain amplification.
Two honest limits:
- How much it helps depends on the source. The gain scales with how closely the original decoder resembled the notch we model. Clean, detailed, fairly static material benefits most; on soft or heavily processed sources the correction is small.
- It is not free on every source. On grainy consumer tape
(VHS/Hi8-class) some of the boosted high frequencies are amplified
grain rather than detail, and on flat or graphic content (titles, test
patterns, color bars) it can overreach. If your source is grainy and
you follow this plugin with an aggressive denoiser, that denoiser
removes most of the amplified grain anyway; if your source is
graphic/flat, set
refine=0.
refine lives inside Restore and cannot be applied later — it is
anchored to the pre-encode picture, which only exists during the round
trip. So the choice is simply whether to enable it in that first
Restore call. Because all of the cross-color / dot-crawl removal (the
plugin's main job) happens independently of refine, setting
refine=0 costs nothing on artifact cleanup; it only forgoes the luma
detail recovery. Leave it at 1 for clean detailed sources; set it to
0 for grainy tape into a denoiser, for graphic/flat content, or
whenever you want a conservative default that adds no high frequencies
of its own.
The defaults are the best general setting and you can stop here. The numbers below are for when you want to tune.
How to read the tables. Every clip is measured as: clean source →
simulated bad decoder → this plugin → compared back to the clean source
(test/readme_tables.py, built on test/metrics.py). degraded is the
artifact-ridden input (doing nothing); transparency is the clean
source run straight through encode/decode with no bad decoder — the PSNR
ceiling (for the lower-is-better columns it is just a reference, not a
floor). chromaHF is residual chroma high-frequency energy (rainbows /
dot patterns) and flicker is frame-to-frame chroma change (crawl);
for both, lower is better, and the % is how much of the artifact
each mode removed versus degraded. PSNR is fidelity to the clean
source in dB (higher is better).
Modes are compared through Decode; Restore adds a Y-only detail
recovery (refine) on top, so under Restore the luma PSNR is higher
and the chroma numbers here are unchanged. Each mode is measured at its
own defaults, so the comb rows use eq=1 (the no-transform default) and
the transform/hybrid rows use eq=2 — part of the comb's higher
chromaHF is that equalization choice, not the separator itself.
Corpora: VQEG rows are held-out clips (src20–22) the shipped
trained tables were not trained on — the generalization number.
BT.802 is the NTSC restoration-target corpus the tables were tuned
on, split into stills (Rec. BT.802 scenes 1–13, single frame — every
temporal metric is trivially flat, so this is the easy case) and
motion (scenes 14+, real footage). The never-trained VQEG held-out
and the tuned-on BT.802-motion rows agree closely (chromaHF −73% vs
−75%; flicker −82% vs −73%, the same ballpark) — the evidence the
separation generalizes. bars is SMPTE EG-1 / EBU color
bars (flat color fidelity) and zone is a zone-plate sweep — a pure
cross-color torture test.
On real footage the defaults remove about 73–75% of the rainbow (chromaHF) and 73–82% of the dot-crawl (flicker). chromaHF / flicker columns are the residual; (−N%) is how much was removed vs. degraded (higher removed % is better). PSNR is dB vs. the clean source.
| corpus | mode | PSNR Y / U / V | chromaHF | flicker |
|---|---|---|---|---|
| VQEG (held-out) | degraded | 30.7 / 32.9 / 35.1 | 419 | 1356 |
| default (3D hybrid) | 33.1 / 39.2 / 39.8 | 115 (−73%) | 249 (−82%) | |
comb 3D (transform=0) |
32.8 / 38.6 / 39.6 | 233 (−44%) | 354 (−74%) | |
transform 3D (transform=1) |
33.0 / 39.0 / 39.6 | 107 (−74%) | 261 (−81%) | |
2D comb (dimensions=2) |
31.7 / 36.0 / 37.7 | 334 (−20%) | 860 (−37%) | |
| transparency | 41.5 / 39.8 / 40.4 | 163 | 302 | |
| BT.802 motion (scenes 14+) | degraded | 30.2 / 32.0 / 34.3 | 581 | 1629 |
| default (3D hybrid) | 32.2 / 37.7 / 39.3 | 147 (−75%) | 434 (−73%) | |
| transform 3D | 32.1 / 38.2 / 39.8 | 138 (−76%) | 402 (−75%) | |
| comb 3D | 31.9 / 36.7 / 38.5 | 298 (−49%) | 548 (−66%) | |
| 2D comb | 31.3 / 35.5 / 37.7 | 369 (−37%) | 1042 (−36%) | |
| transparency | 40.8 / 38.6 / 40.4 | 213 | 520 | |
| BT.802 stills (scenes 1–13) | degraded | 28.5 / 30.2 / 32.2 | 606 | 1924 |
| default (3D hybrid) | 31.4 / 37.9 / 38.5 | 158 (−74%) | 136 (−93%) | |
| comb 3D | 31.4 / 39.2 / 39.8 | 180 (−70%) | 17 (−99%) | |
| 2D comb | 29.5 / 33.3 / 34.9 | 391 (−35%) | 1203 (−37%) | |
| transparency | 40.8 / 40.0 / 40.7 | 212 | 170 | |
| bars (SMPTE EG-1) | degraded | 40.5 / 32.9 / 36.5 | 1186 | 110 |
| default (3D hybrid) | 40.7 / 36.5 / 37.8 | 218 (−82%) | 217 (—) | |
| comb 3D | 42.6 / 37.9 / 39.5 | 218 (−82%) | 1 (—) | |
| transparency | 44.0 / 41.0 / 41.8 | 223 | 225 | |
| zone (torture test) | degraded | 23.3 / 27.9 / 30.9 | 655 | 2377 |
| default (3D hybrid) | 26.2 / 87.8 / 94.2 | 0 (−100%) | 0 (−100%) | |
| 2D comb | 24.9 / 34.3 / 37.2 | 432 (−34%) | 1265 (−47%) | |
| transparency | 68.1 / 90.3 / 96.3 | 0 | 0 |
On the bars rows the flicker column is round-trip residual, not real crawl (a static frame has no motion — its only frame-to-frame change is the subcarrier sequence), so no reduction % is meaningful there; read flicker on the motion corpora. The comb nulls that residual on a static input, which is why it reads near-zero on stills and bars — but that does not hold on real footage, where the hybrid wins.
On held-out footage the defaults remove about 43% of the rainbow and
26% of the dot-crawl; evidence=1.0 pushes both a little further.
| corpus | mode | PSNR Y / U / V | chromaHF | flicker |
|---|---|---|---|---|
| VQEG (held-out) | degraded | 35.4 / 37.6 / 39.2 | 222 | 1366 |
| default (3D) | 35.8 / 38.2 / 38.0 | 127 (−43%) | 1012 (−26%) | |
default +evidence=1.0 |
35.7 / 38.0 / 37.6 | 122 (−45%) | 971 (−29%) | |
trained 2D (dimensions=2) |
35.8 / 37.8 / 37.9 | 129 (−42%) | 1053 (−23%) | |
level 2D (level=1) |
35.6 / 38.2 / 38.2 | 146 (−34%) | 1105 (−19%) | |
threshold 2D (threshold=0.4) |
35.6 / 38.3 / 38.4 | 155 (−30%) | 1153 (−16%) | |
| transparency | 41.1 / 38.6 / 38.5 | 149 | 1085 | |
| bars (EBU) | degraded | 46.8 / 36.8 / 39.1 | 430 | 191 |
| default (3D) | 48.0 / 37.4 / 39.9 | 210 (−51%) | 89 (−53%) | |
| trained 2D | 49.3 / 37.5 / 39.9 | 212 (−51%) | 3 (−98%) | |
| transparency | 52.8 / 44.5 / 48.9 | 229 | 90 | |
| zone (torture test) | degraded | 25.7 / 38.0 / 41.0 | 440 | 599 |
| default (3D) | 26.2 / 76.3 / 79.9 | 1 (−100%) | 4 (−99%) | |
| trained 2D | 26.1 / 74.9 / 79.4 | 1 (−100%) | 6 (−99%) | |
| transparency | 56.7 / 69.3 / 72.6 | 2 | 7 |
Guidance:
- Just use the defaults (
dimensions=3, trained tables, NTSCtransform=2). They win or tie almost everywhere. dimensions=2is the fast path — no neighboring frames, several times less compute. Use it for stills, very short clips, or previews.- NTSC
transform=picks how the 3D path separates:0a comb (near-exact on fully static content),1a Transform separator (stronger on motion),2(default) routes between them per sample. - Artifact-heavy footage: add
evidence=1.0(PAL) to knock down residual chroma flicker in flat regions. - Graphics / titles / test patterns: try
cti=1(chroma transient improvement) to re-sharpen color edges; leave it off on natural footage, where it hurts.
out = core.composite.Decode(comp, standard="ntsc", width=720)Decodes an already-composite GRAY16 clip (the output of Encode) back
to YUV444P16. Restore calls this internally; use it directly only if
you are working with composite signals yourself. Composite input must be
exactly 758×480, 758×486 (NTSC) or 928×576 (PAL).
Parameters (all optional, all also available on Restore):
standard—"pal"or"ntsc". Required to match the encode.width— output width, default720. Decoding happens internally on the 4×fsc raster and the result is resampled towidthwith a subpixel crop that lands the samples on the BT.601 grid (the exact inverse ofEncode's mapping), so a round trip is geometry-preserving. Setwidth=0to skip that final horizontal resample entirely and get the raw decode raster — NTSC 758, PAL 928 wide (the SMPTE 244M / EBU 4fsc sampling), at the input height. The picture (and themaskoutput, which stays crisp) then come straight off the decode grid with no resize; these are non-square-pixel frames you resample yourself. Note that settingwidthto the raster value (e.g.758) is not the same — that still resamples; onlywidth=0bypasses it.dimensions—3(default) spatio-temporal separation using neighboring frames;2fast 2D (spatial / line comb);1a crude notch reference (worst case).transform— NTSCdimensions=3only:0comb,1Transform,2motion-routed hybrid (default).eq— chroma equalization.2(default on Transform paths) steers chroma bandwidth by the separator's own confidence;1a fixed inverse filter;0off.evidence— PAL only, default0. A low-frequency luma prior that attenuates chroma with no luma partner;0.5–1cuts flicker on artifact-heavy footage.cti— luma-guided chroma transient improvement, default off. For graphics-like sources only.setup— NTSC 7.5 IRE pedestal; must match the encode.mask— output a per-sample mask as a second clip:"motion"(the NTSC hybrid router) or"confidence"(the separation confidence). See Masks below.
Advanced separation controls (threshold, thresholds, level, lut)
override the built-in trained tables; see the comments in
test/calibrate_thresholds.py. level=1 is the robust untrained
alternative to the trained tables on synthetic extremes.
Decode and Restore can output a per-sample mask as a second clip
alongside the picture, so you can postprocess selected regions
differently — the region a sample fell into, or how well it separated.
Two kinds are available via mask=.
The mask comes from the same decode as the picture (no second pass) and
is resampled to the output width with bilinear — a clean soft edge,
unlike the picture's sharper filter. For a crisp, un-resampled mask use
width=0 (see width above); the mask then comes straight off the
decode raster.
How the two outputs are returned differs by host:
VapourSynth returns a two-element list, [picture, mask]:
pic, mask = core.composite.Restore(clip, standard="ntsc", mask="motion")
alt = pic.some.AggressiveChromaCleanup()
out = core.std.MaskedMerge(pic, alt, mask, planes=[1, 2])AviSynth+ returns one YUVA clip with the mask as its alpha; pull it
out with ExtractA:
dec = composite_Restore(clip, standard="ntsc", mask="motion")
mask = ExtractA(dec)
alt = AggressiveChromaCleanup(dec)
Overlay(dec, alt, mask=mask)
On the NTSC hybrid path (the default: dimensions=3, transform=2) the
decoder routes each sample to the comb (still regions) or the Transform
separator (motion). mask="motion" exposes that per-pixel decision:
white = motion, black = still — matching the mvtools/mvutensils
convention, so MaskedMerge processes the moving regions. It works only
on the NTSC hybrid path (it errors elsewhere, since no other path has a
motion router).
mask="confidence" exposes the separator's per-sample confidence — the
same signal behind the CompositeSeparationConfidence* properties —
available on any eq=2 path (all Transform separations, PAL and
NTSC; it errors when eq is not 2). Unlike the binary motion mask it is
soft (a graded 0–max mask). Polarity is white = least confident,
i.e. the inverse of the confidence property (a PlaneStats mean of the
mask is ≈ 1 − CompositeSeparationConfidenceMean): white marks the
samples where chroma separation was most suspect — the ones you would
clean hardest.
width=0 gives you the raw 4×fsc raster with no horizontal resample, so
you can run your own processing at that sampling and convert to BT.601
later without a double resize. When you do want BT.601 (720-wide,
square-ish pixels), reproduce exactly what the plugin does internally:
replicate-pad the edges, then a subpixel-crop Spline36. This is
byte-identical to Decode(width=720):
def to_bt601(raw, standard, width=720):
# rho = 4fsc/13.5 MHz sample-rate ratio; active0 = active-window start
# on the 4fsc raster; anchor601 = BT.601 first active luma sample
if standard == "pal":
rho, active0, anchor601 = 540000 / 709379, 182.0, 132.0
else:
rho, active0, anchor601 = 33 / 35, 130 + 57 / 90, 122.0
pad = 24
# edge-replicate `pad` columns each side (NOT black — a black step
# would make Spline36 ring inward along the frame border)
left = raw.std.Crop(right=raw.width - 1).resize.Point(width=pad)
right = raw.std.Crop(left=raw.width - 1).resize.Point(width=pad)
padded = core.std.StackHorizontal([left, raw, right])
src_left = pad + anchor601 / rho - (active0 - 0.5) - 0.5 * (720 / width) / rho
return padded.resize.Spline36(width=width, height=raw.height,
src_left=src_left, src_width=720 / rho)Edges are filled by replication here; the plugin's own Restore instead
passes the few outermost columns through from the source (they sample
beyond the raster and were never reconstructed). Reproduce that only if
you specifically want byte-identical-to-Restore borders and still hold
the original source clip — for most processing, the replicated edge is
fine (and more consistent).
Decode and Restore tag each output frame with a few read-only
diagnostics — how hard the decode was, per frame. They are difficulty
signals, not quality scores: there is no clean reference to score
against during restoration, so these report the decoder's own effort and
confidence, which correlate with where artifacts are likely to remain.
Read them in a script to log, plot, or gate later processing (for
example, denoise harder on low-confidence frames).
Each property appears only when the path that produces it is active,
so its presence is itself informative. In VapourSynth they are on
frame.props; in AviSynth+ read them with propGetFloat.
| Property | Present when | Meaning |
|---|---|---|
CompositeSeparationConfidenceMean |
eq=2 (the default on transform paths) |
Mean of the separator's per-sample confidence, ~0–1. Near 1 where chroma separated cleanly; low where luma leaked into chroma (rainbow-prone content). Lower = a harder frame. |
CompositeSeparationConfidenceStdDev |
eq=2 |
Spread of that confidence across the frame. High std means the trouble is localized (a few bad regions) rather than uniform. |
CompositeMotionFraction |
NTSC dimensions=3, transform=2 (the default) |
Fraction of samples the motion router judged to be in motion (and sent to the transform), 0–1. Near 0 = a nearly still frame (the comb handled it); near 1 = mostly motion. |
CompositeRefineResidual |
Restore with refine>0 (the default) |
Mean luma the crude-decoder model still cannot reproduce after refinement. High = the source's original decoder was unlike the model, so refine could only partly fit it. |
CompositeRefineCorrection |
Restore with refine>0 |
Mean amount refine moved the luma. Large = a lot of softened detail was recovered. (Equal to the residual at refine=1; they diverge at higher counts as the residual falls and the total correction grows.) |
All values are per frame and computed over the active picture. Absent
properties simply mean that path was not taken (for example, no
CompositeMotionFraction on PAL, or no refine properties at refine=0).
comp = core.composite.Encode(clip, standard="ntsc")The forward direction: YUV to a composite GRAY16 clip on the 4×fsc
active raster (PAL 928×576, NTSC 758×480/486). Mostly useful for making
test composites; Restore does the encode for you.
standard—"pal"or"ntsc".setup— NTSC only: add the 7.5 IRE pedestal (default off).precomb— vertically low-pass U/V before modulation (Poynton's precombing). Helpful only when feeding a comb decoder a clean source; leave it off for cleanup, where it measurably hurts.
Some chroma residue is genuinely modulated color that no spectral separator can touch — but it is phase-incoherent along motion (rainbows rotate frame to frame while real color stays put). A motion-compensated chroma degrain after the decode cancels exactly that, with vapoursynth-mvutensils:
dec = core.composite.Restore(clip)
sup = core.mvu.Super(dec, blksize=16, overlap=8, pel=2)
vec = core.mvu.AnalyseMany(sup, radius=2)
out = core.mvu.Degrain(dec, sup, vec, planes=[1, 2], thsad=[400, 1600])Vectors come from the already-clean decoded luma; only chroma is
touched; where vectors fail it falls back to the unprocessed pixel.
Strength saturates around thsad=[400, 1600], radius=2. Keep it after
the decode — the round trip must see the artifacts untouched.
comp = core.composite.Encode(clip, standard="ntsc", precomb=1)precomb=1 nulls line-alternating chroma exactly — the friendly choice
when the consumer is a comb decoder. Keep it off for cleanup.
VapourSynth: drop composite.so/composite.dll in your plugins
autoload dir (or core.std.LoadPlugin(...)).
AviSynth+: load the same module. The AviSynth frontend registers
composite_Encode, composite_Decode, composite_Restore, and needs
avsresize (z_ConvertFormat) loaded at runtime for the internal
resampling — install it alongside.
meson setup build
ninja -C build
meson test -C buildRequires Meson, a C99 compiler, FFTW3 (single precision), and
VapourSynth (V4 API) with the Python module available for header
discovery. Add -Davisynth=true to also build the AviSynth+ frontend
into the same module (the release wheels enable it).
End-to-end plugin tests (run by hand; each needs its host at runtime):
# VapourSynth
python test/test_composite.py build/composite.soThe AviSynth+ end-to-end test is a C harness that links libavisynth, so
it is an opt-in build target — point -Davisynth_lib_dir at the
directory holding your libavisynth.so.N:
meson setup build -Davisynth=true -Davisynth_lib_dir=$LIBAVS
ninja -C build
# run against the built module and a working avsresize:
./build/test_composite_avs build/composite.so /path/to/avsresize.soIt links libavisynth to build and needs a working avsresize
(z_ConvertFormat) at runtime for the resampling paths; without one it
still runs the argument-validation checks. Note some avsresize builds
export AVS_linkage as a global and segfault on load — that is an
avsresize build issue (relink it with -Wl,-Bsymbolic), not this plugin.
Round-tripping Y'CbCr through a composite encode and a good decode removes cross-luma and cross-color artifacts baked in by a bad hardware decoder: the re-encode reconstructs the composite signal the bad decoder saw, and a Transform/comb decode re-separates it properly. The signal processing follows ld-decode's ld-chroma-encoder and ld-chroma-decoder, after Clarke, Colour encoding and decoding techniques for line-locked sampled PAL and NTSC television signals, BBC RD 1986/2, with separation modes from GB 2365247 A (Easterbrook) and US 7,872,689 (Weston), and decoder adaptivity after Faroudja (NTSC and Beyond, 1988).
GPL-3.0-or-later. See COPYING.