fix: require transformers>=5.4 for local backend (#83)

[ssrajadh]

Fixes #83.

The bug

transformers 5.3.0 dropped the per-frame expansion of video_grid_thw in Qwen3VLModel.get_rope_index. The Qwen3-VL processor interleaves <t seconds> text between per-frame <|vision_start|>…<|vision_end|> blocks, and create_mm_token_type_ids types only the video tokens themselves — so an N-frame chunk produces N contiguous video runs. Each run calls next() on an iterator over video_grid_thw, which holds one row per video. The iterator exhausts after the first frame:

File "transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1126, in get_rope_index
    grid_thw = next(grid_iters[modality_type])
StopIteration

Every chunk fails and lands in the DLQ: Indexed 0 new chunks from 0 files (4 failed → DLQ).

Only 5.3.0 is affected. 5.0–5.2 and 5.4.0+ all expand the grid with repeat_interleave(video_grid_thw, video_grid_thw[:, 0], dim=0) before iterating. Verified directly against the upstream sources for 5.0.0, 5.1.0, 5.2.0, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0 and 5.14.1.

Why it went unnoticed

uv.lock has pinned exactly 5.3.0 since the uv migration (2c051a9, 2026-03-26 — 5.3.0 was the newest release that day; 5.4.0 shipped on 3/27). 60b84dc then removed the inputs.pop("mm_token_type_ids", None) workaround on 3/28, which is what turned the upstream bug into a hard failure — in 5.3.0, compute_3d_position_ids skips get_rope_index entirely when mm_token_type_ids is absent, so the pop had been suppressing this crash. A fresh resolve on 3/28 already got the fixed 5.4.0, so the change tested clean.

The result is a split by install path, not by platform:

• uv tool install ".[local]" (README) ignores the lock, resolves fresh, has been on 5.4+ since March — fine.
• uv sync [--frozen] --extra local (CONTRIBUTING.md:18, CI) reads the lock, gets 5.3.0 — broken on every OS.

Plain uv sync doesn't rescue you either: 5.3.0 satisfies >=5.3, so the lock is never considered stale and is never re-solved.

check-local-deps runs on ubuntu/macos/windows against the lock, but only does import LocalEmbedder; print('OK') — it has been installing the broken version on all three and passing, because importing never builds position ids.

The change

• Raise the floor to transformers>=5.4 in the local and local-quantized extras, with a comment, so 5.3.0 can never be resolved again.
• uv lock --upgrade-package transformers → 5.14.1.

Lockfile delta is exactly two packages: transformers 5.3.0 → 5.14.1 and safetensors 0.7.0 → 0.8.0. torch / torchvision / torchcodec pins are untouched, so the Windows CUDA-wheel handling (d28520d) and the torchvision GPU pin (6fda034) are unaffected.

Testing

• 308 unit tests pass.
• uv lock --check clean.
• Confirmed TRANSFORMERS_DISABLE_TORCH_CHECK — the MPS escape hatch in local_embedder.py:242-245 — still exists and is still honored in 5.14.1 (utils/import_utils.py:1640), and the "Video features and video tokens do not match" check still routes through it.

⚠️ Needs a smoke test on Apple Silicon before merge. This moves 11 minor releases within the 5.x line, and I could not exercise the MPS path here. check-local-deps going green will not cover it — one real sentrysearch index run on a Mac would.

Follow-up (not in this PR)

The retry handler formats {exc} (cli.py:164, :172), and StopIteration carries no message, so the reporter saw Embed error (attempt 1/3), retrying in 2s: with nothing after it. Worth switching to {exc!r}. Relatedly, _is_permanent_failure (cli.py:114) matches on str(exc).lower() and so can never classify a message-less exception, burning all 3 attempts.

🤖 Generated with Claude Code