rkvdec-slice-lab

Can a hardware decoder handle H.264/H.265 video with missing slices?

Short answer: yes — on both stacks tested, the hardware handles it fine. It decodes the surviving slices and conceals the hole. What breaks is the software above it, and it breaks in two completely different ways: Rockchip MPP stops submitting work to the hardware and freezes on the last good frame, while ffmpeg's VAAPI path throws away a perfectly good GPU surface because the driver flagged it. Both look like flaky silicon. Neither is.

Measured on Rockchip RK3588 (vendor MPP) and Intel N100 (VAAPI/iHD), fed byte-identical streams from a real camera. This repo contains the testbench, the measurements, and step-by-step reproduction. Everything here was measured, not inferred.

---

The claim under test

"rk3566 / VAAPI hw decoders does not accept partial/missing/corrupt slices"

Several separate claims are tangled together there. They land differently:

Claim	Verdict	Evidence
Rockchip can't decode sliced video	False	8-slice H.264 and 4-slice H.265 from a real SigmaStar camera decode on RKVDEC bit-identical to libavcodec, 161/161 frames, zero differing pixels
Rockchip can't decode video with a missing slice	False	RKVDEC decodes the survivors and conceals the gap. Damage stayed inside the lost slice ±1 deblocking row; 2.81% of pixels wrong; PSNR 38 dB
VAAPI can't decode video with a missing slice	False	the Intel GPU decoded all 161 frames and concealed the hole at 69 dB, damage confined to the lost slice
...and yet the stream dies on both	True on both — two unrelated software bugs	Rockchip: MPP stops submitting to hardware. VAAPI: ffmpeg refuses a surface the driver flagged, at the first CPU readback.

Same bytes, both stacks, mirror-image symptoms:

	Rockchip MPP	Intel VAAPI
H.264 slice loss	nondeterministic freeze latch	deterministic fatal at readback; GPU picture is perfect
H.265 slice loss	silent corruption, smears past the slice, errinfo 0	clean, contained, no error
slice-replay repair	helps a lot	no-op (H.264) or actively harmful (H.265)

Remaining scope limit: the mainline V4L2-stateless rkvdec driver — what libva-v4l2-request / GStreamer v4l2codecs use on RK356x — is a third implementation again and was not tested. Given how far the two measured stacks diverge on identical input, don't assume it matches either. See docs/FINDINGS.md §10.

---

The real defects — Rockchip MPP (1-3), ffmpeg VAAPI (4)

1. A nondeterministic freeze latch (H.264). A race between MPP's parser thread and its hardware-done callback. When the parser loses, MPP marks the frame ref_err, skips hardware submission entirely, and returns stale buffers — forever, until the next IDR. Proven by the output repeating bit-for-bit with the buffer-pool period.

   It is triggered by consumer back-pressure. Same file, same binary:

   consumer idle                     : 4  4  4  4  4  4
   consumer touches every frame      : 4 39 41 18 32 46     <- realistic
   consumer + base:disable_error=1   : 5  5  5  5  5  5     <- fixed
   

2. Silent corruption (H.265). Drop a slice segment and errinfo stays 0 on every frame while ~13% of the picture is wrong and the damage smears well past the lost slice. MPP's HEVC parser never checks slice_segment_address continuity.

3. errinfo is not trustworthy. On a real camera stream, 32 frames were flagged while 60 were actually damaged — and two of four identical loss events latched while two didn't. Do not use mpp_frame_get_errinfo() to decide whether a frame is usable.

4. A good GPU surface thrown away (VAAPI). The Intel GPU decodes the damaged stream completely — 161/161 frames, hole concealed at 69 dB. But vaSyncSurface() returns VA_STATUS_ERROR_DECODING_ERROR (23) for the affected surface, libavutil maps that to AVERROR(EIO), and ffmpeg makes it fatal at the first CPU readback:

   [AVHWFramesContext] Failed to sync surface 0xa: 23 (internal decoding error).
   [h264] Failed to transfer data to output frame: -5.
   ... Conversion failed!
   

   In the same run ffmpeg reports "25 frames decoded; 0 decode errors" — the decoder never complained, only the surface sync did. Deterministic, 5/5 runs. Decode with -f null (no readback) or re-encode on-GPU and all 161 frames come through.

---

Do you actually need a bitstream repair? Probably not

Both failures are software policy and both are fixable — see docs/FIXES.md, with patches in patches/.

	fix the software	replay the lost slice instead
Rockchip MPP	base:disable_error=1 — one config line, no patch. Stops the freeze: 5 4 5 5 5 5 under load vs 33 18 46 46 32 46. Damage left at the hardware's own concealment, 38 dB, confined to the lost slice	also no freeze, and fills the hole better: 63.5 dB
Intel VAAPI	small libavutil patch. 161/161 frames at 69.9 dB	no-op on H.264, −20 dB on H.265

They are not alternatives on Rockchip — the fix stops the catastrophic failure (a frozen stream), the repair improves the quality of the surviving glitch. Doing both is better than either; if you can only do one, do the fix. On VAAPI the fix is strictly better and the repair does nothing useful.

On VAAPI, no existing ffmpeg flag helps (-fflags +output_corrupt, -flags2 +ignore_err, -err_detect ignore_err, -hwaccel_flags +ignore_err/+unsafe_output all still abort at frame 25 of 161) because the failure is below the codec layer. tools/vasync_shim.c proves the fix with an LD_PRELOAD before you patch anything: downgrading only VA_STATUS_ERROR_DECODING_ERROR turns 25 frames into 161 at 69.9 dB.

So the honest case for a repair layer is narrow: "I ship a decoder stack I cannot change." Otherwise fixing the error policy is smaller, portable, and gives a better picture.

---

If you are building a NAL-repair loop anyway

Substituting a lost slice with the previous frame's slice at the same position works, and beats every alternative measured:

first damaged frame, H.264	PSNR
replay previous frame's slice	36.7 dB (synthetic) / 64.1 dB (real camera)
ideal temporal copy of the region (computed ceiling)	31.6 dB
slice simply dropped	29.4 dB
slice truncated to 50% and forwarded	28.5 dB — and it corrupts the following slices

Because replay beats an ideal frame-copy, don't build a synthesised mb_skip_run concealment slice; it would be strictly worse and, with CABAC, much harder.

It does not port as-is. Raw slice replay works on MPP only because MPP's frame-boundary detection ignores frame_num. libavcodec validates it and rejects the replayed slice outright (Frame num change from 5 to 4 / decode_slice_header error), making the repair a no-op on H.264 — and on HEVC it accepts the slice and lands 20 dB worse than simply dropping it (42.95 dB vs 62.97 dB). To be portable the substituted slice header must be rewritten with the current picture's frame_num, pic_order_cnt_lsb and reference-marking fields, not merely copied. See docs/FINDINGS.md §9.2.

Five rules that fall out of the measurements:

1. Never substitute slice 0. It carries the picture-level parameters (frame_num, POC, dec_ref_pic_marking). Replaying the previous frame's slice 0 makes MPP build the previous picture's parameters: quality collapses to 25 dB from the very next frame. Drop the whole access unit instead, or rewrite those header fields.
2. Feed one complete access unit per packet with base:split_parse=0.
3. Set base:disable_error=1 (MPP_DEC_SET_DISABLE_ERROR). It alone removes the freeze latch. FFmpeg's rkmppdec.c does not set it and treats any flagged frame as fatal AVERROR_UNKNOWN.
4. Drop truncated slices, never forward them. A corrupt slice is worse than an absent one and spreads damage into the following slices of the picture.
5. Your intra-refresh period is your worst-case recovery time. Measured: 30-frame refresh period → pixel-exact recovery 24 frames after the loss; 300-frame period → no visible recovery within 70 frames.

---

Quickstart

Two machines: a decoder board (RK3588, vendor kernel, /dev/mpp_service) and optionally a SigmaStar camera running OpenIPC/majestic.

git clone <this-repo> ~/rkvdec-slice-lab && cd ~/rkvdec-slice-lab

# on the decoder board
./scripts/00-build-mpp.sh          # clone + build MPP, build slice_probe
./scripts/10-make-corpus.sh        # synthetic multi-slice corpus via ffmpeg
./scripts/20-synthetic-matrix.sh   # the slice-loss matrix
./scripts/30-latch-race.sh         # the freeze latch, its trigger, and the fix
./scripts/70-mpp-fix.sh            # fix vs bitstream repair, head to head

# optional: real camera end-to-end
CAMERA=192.168.1.10 CAMERA_PASS=secret ./scripts/40-camera-slices.sh
./scripts/50-camera-matrix.sh

# on an Intel/AMD box with VA-API — the other half of the claim
./scripts/60-vaapi-matrix.sh                                  # H.264
STREAM=sstar_slice.h265 CODEC=h265 SLICE=2 ./scripts/60-vaapi-matrix.sh

No camera? samples/ has 2 s of real SigmaStar output, 8-slice H.264 and 4-slice H.265 at 2560x1920 — copy them into $CORPUS as sstar_slice.h264 / sstar_slice.h265.

Full environment details, versions, and the traps: docs/REPRODUCE.md.

---

Layout

CLAUDE.md           orientation for coding agents: rules, dead ends, conventions
docs/REPRODUCE.md   step-by-step environment setup, versions, known traps
docs/FINDINGS.md    every measurement, with the exact numbers
docs/MECHANISM.md   source-level analysis of MPP, with file:line citations
docs/FIXES.md       how to fix the software instead of repairing the bitstream
patches/            ffmpeg patches for the VAAPI and rkmpp error policies
tools/nalcut.py     Annex-B slice surgery: list/drop/repair/trunc/dropau/aud
tools/slice_probe.c MPP decode harness that reports errinfo and dumps every frame
tools/compare_yuv.py per-frame PSNR + damage localised to macroblock rows
tools/freeze_check.py detects the freeze latch via bit-exact buffer periodicity
tools/vasync_shim.c  LD_PRELOAD proving the VA surface ffmpeg discards is good
scripts/           numbered; 00-50 Rockchip, 60 VA-API (different machine)
results/           raw logs from the runs the docs quote, incl. results/vaapi/
samples/           2 s of real SigmaStar multi-slice output

Enabling slices on an OpenIPC SigmaStar camera

No firmware change needed — majestic already implements it (src/sstar/sdk.c → MI_VENC_SetH264SliceSplit / MI_VENC_SetH265SliceSplit):

curl "http://root:PASS@CAM/api/v1/set?video0.sliceUnits=16"   # then restart majestic

Two gotchas: sliceUnits is macroblock rows per slice, not slice count, and it is clamped to 0..16. It changes a channel-create attribute, so majestic must be restarted, not reloaded. At 2560x1920 with sliceUnits=16 you get 8 slices for H.264 and 4 slice segments for H.265.

Status / caveats

• Measured 2026-08-20 against MPP develop @ c08762eb, RK3588 vendor kernel 6.1.115, RKVDEC HW_ID 0x53813f05 (HWID_VDPU38X, served by the vdpu34x HAL).
• VAAPI side: Intel N100 (Alder Lake-N), iHD 25.2.3, VA-API 1.22, ffmpeg 7.1.5.
• The mainline V4L2-stateless rkvdec path is untested.
• Not upstreamed to Rockchip. The disable_error=1 workaround is userspace-only and needs no patch.
• The scripts were run from a clean checkout on a wiped board to confirm they reproduce the documented numbers — transcript in results/verification-run.md.

MIT licensed — see LICENSE.