Weekend Mobile Tests (LLM) — 2026-07-12

[github-actions]

Automated weekend mobile run for @qvac/llm-llamacpp.

Overall: ❌ failures detected

Platform	Result	Total	Passed	Failed
Android	❌ failed	36	34	2
iOS	✅ passed	24	24	0

Full per-test tables and Device Farm run links are on the workflow run summary.

Weekly coverage: OCR (LightOn + Paddle), Elephant + HighResAurora image functional, and vlmPerf{Gemma4,Qwen35}.

[olyasir]

Root cause found. The 2 Android failures are the same test — runGemma4ImageElephantPerfTest — crashing the app (SIGABRT) on Samsung Galaxy S25 Ultra and S26 Ultra.

Backtrace (identical on both devices): ggml_abort → ggml_cl_compute_forward → clip_image_batch_encode → mtmd_encode — the Gemma 4 vision projector aborting on the Adreno OpenCL backend.

Why now: #3162 (QVAC-21867) + #3168 (merged Jul 8–9, llm-llamacpp 0.36.x) changed the Android mmproj default from CPU to GPU on Adreno 800+. Device log 1.5 s before the crash: [LlamaModel] multimodal projector backend: GPU (auto-default, Adreno 800+). The Jul 5 weekend run (pre-#3162) passed the identical suite.

Why only Gemma 4: its mmproj is bf16 (mmproj-google_gemma-4-E2B-it-bf16.gguf); the OpenCL backend has no bf16 kernels for at least one op in its vision graph. On the exact same Samsung GPU path, f16 (Qwen3.5) and Q8_0 (SmolVLM2, PaddleOCR-VL) projectors pass. Pixel 9 Pro is Mali → projector stays on CPU → passes; iOS unaffected.

Actions:

• CI mitigation: QVAC-22256 test[mod]: use f16 Gemma4 mmproj in VLM tests (bf16 aborts on Adreno OpenCL) #3255 switches the suite to the f16 mmproj.
• Product bug (bf16 projector is still served by the prod registry / SDK as MMPROJ_GEMMA4_2B_MULTIMODAL_BF16, so real Adreno 800+ users can still hit this): tracked in QVAC-22256 — OpenCL supports_op should reject the bf16 op (CPU fallback via scheduler) or the addon auto-default should keep bf16 projectors on CPU.