Add resumable generation checkpoints

[ArthurOstapenko]

Summary

• pause a full Euler denoising schedule after a step count or denoising wall-time threshold
• save video and audio sampler latents atomically, with strict resume compatibility checks
• decode a sigma-zero draft from a copy of the paused state without another DiT forward
• expose pause/resume through the C API and CLI, with tests and usage documentation

Why

A short completed run cannot be extended: its schedule has already reached sigma zero. Progressive checkpoints instead pause the intended final schedule, so users can inspect composition and motion before deciding whether to continue the remaining DiT forwards.

Design

The checkpoint keeps the unfinished F32 video/audio sampler state. Draft decoding operates on a separate sigma-zero projection, so inspecting a draft never mutates the state used for continuation.

Checkpoint writes use a temporary mode-0600 file, fsync, and atomic rename. The format checks its header and payload, rejects truncated or corrupted files, and validates the prompt, model directory, seed, denoising schedule, internal render shape, references, and semantic quality settings before resume.

Progressive v1 is limited to --reuse 1 --core-reuse 1; accelerated reuse paths carry history that is not stored in the checkpoint.

On resume, --checkpoint-after-step remains absolute within the original schedule, while --checkpoint-after-seconds measures denoising time in the current invocation. Reusable decode-only VAE latent artifacts are out of scope here and can be added separately.

Example

./h3 -d ./MiniMax-H3 -p "$PROMPT" \
  --width 512 --height 512 --frames 22 \
  --steps 20 --layers 50 --reuse 1 --core-reuse 1 \
  --checkpoint-after-step 10 \
  --checkpoint outputs/progress.h3ckpt \
  -o outputs/draft.mp4

./h3 -d ./MiniMax-H3 -p "$PROMPT" \
  --width 512 --height 512 --frames 22 \
  --steps 20 --layers 50 --reuse 1 --core-reuse 1 \
  --resume outputs/progress.h3ckpt \
  -o outputs/final.mp4

Validation

• clean build of h3, h3_tests, h3_semantic_vae_test and h3_real_video_vae_test, with no compiler warnings
• ./h3_tests: 1846 checks, including the Metal probe
• checkpoint create/replace/load round trip
• compatibility mismatch rejection
• truncated, extended, malformed, corrupted and symlinked checkpoint rejection
• sigma-zero Euler projection equivalence
• Clang static analysis of the checkpoint, API and DiT paths
• C++17 public-header compile smoke