Nonblocking MVP experiment: sanitized KV-Ground smoke on external shared GPU

[mark-e-deyoung]

Purpose

Exploit third-party/shared GPU resources to falsify public model/runtime assumptions early without adding a new dependency to the first-consumer MVP critical path.

This issue implements the external-compute lane from the red-team note on #18. It is explicitly opportunistic and may be stopped if platform setup/quota friction costs more than the uncertainty it removes.

Hypothesis

Given:

• the exact public KV-Ground artifact revision already captured by Foundry;
• a deliberately public/sanitized GUI-grounding fixture;
• a compatible Transformers/PyTorch runtime;

…the model can load and produce a parseable grounding response on a third-party shared GPU without any desktop-ui-cv private source, private screenshots, provider credentials, or target-specific evidence.

A failure is useful evidence if it identifies a model/runtime/preprocessing/output-shape incompatibility before the authoritative local target run.

Why this is not target qualification

External shared hardware is not equivalent to the real desktop-ui-cv target. Results from this issue must not qualify:

• target peak VRAM;
• target model-load latency;
• target p95 query latency;
• private real-workload quality;
• Windows/local compatibility;
• the first real Foundry qualification subject.

Those remain local/private evidence.

Candidate execution planes

Prefer the lowest-friction currently available free/shared GPU. As of the September 2026 red-team, Hugging Face ZeroGPU is a promising option because the service documents shared 48 GB GPU slices for free accounts, but it has platform-specific Gradio/PyTorch/quota constraints and must not become an architectural dependency.

Colab/Kaggle or another available public notebook/shared GPU are acceptable substitutes if they can run the same bounded smoke more cheaply. Do not build platform abstraction to switch among them.

Public-safe inputs only

Allowed:

• exact public model ID/revision;
• public model files/provider-native cache;
• synthetic or deliberately public GUI screenshot;
• public prompt/expected output-shape assertion;
• public runtime/version evidence.

Forbidden:

• WineBot #121 retained/private frames;
• project private holdout/annotations;
• private target inventory;
• desktop-ui-cv private repository source unless separately reviewed for publication;
• tokens/secrets when the public model can be accessed anonymously;
• unpublished failure corpus.

Smallest useful smoke

1. pin exact immutable KV-Ground revision;
2. record runtime/Python/PyTorch/Transformers/GPU environment;
3. load processor + model;
4. run one or a few sanitized GUI-grounding requests;
5. retain raw textual output plus a minimal structural/parser assertion;
6. record load success/failure and bounded elapsed time as environment evidence only;
7. shut down/release shared compute.

Do not run a broad benchmark suite merely to satisfy this issue.

Stop conditions

Stop and proceed with the authoritative local experiment if:

• free/shared GPU access is unavailable or queues make execution impractical;
• platform-specific packaging/Gradio adaptation becomes more work than the smoke itself;
• exact model/runtime cannot fit within the provider's constraints without changing the representation under test;
• solving the external environment starts creating a new deployment product.

Failure to execute this external smoke is not an MVP failure.

Critical-path relationship

This issue runs in parallel with:

• desktop-ui-cv MVP request/result contract for first ui-grounding consumer #35 local-handle seam;
• WineBot #121 real semantic capture;
• finalization of the project resource/performance measurement procedure.

It must not block:

exact identity -> materialization -> frozen request -> local handle -> private holdout -> real target qualification -> receipt -> equivalent repeat -> changed envelope -> GO/NARROW/KILL.

R4 / evidence

Prefer one portable script/notebook/command sequence over platform-specific infrastructure. Record exact source revision and environment. No persistent service is required.

Exit

Either:

• retain a concise PASS/FAIL external execution record and feed any discovered compatibility fact into the authoritative run plan; or
• close/defer with platform friction > uncertainty reduction and continue the local target path.

Related: #18, #29, #35; draft PRs #30/#36/#32; SemperSupra/desktop-ui-cv#35, SemperSupra/WineBot#121.

[mark-e-deyoung]

High-value smoke refinement: test the actual legacy loader assumption

The public model metadata gives this external lane a concrete falsifiable question rather than a generic "does the model run?" smoke.

Observed public artifact metadata:

• exact model config identifies architecture Qwen3VLForConditionalGeneration / model type qwen3_vl;
• current Hugging Face usage guidance for this model uses the multimodal Transformers auto-model path;
• desktop-ui-cv Increment 0 characterized the legacy private sidecar as using AutoModelForCausalLM with trust_remote_code=True.

Do not infer from documentation alone that the legacy path is broken. Test it.

Refined external experiment

Using the same exact immutable KV-Ground revision and same sanitized/public grounding fixture:

1. attempt the legacy-equivalent loader path used by desktop-ui-cv (same model class/kwargs semantics, no private source required);
2. record load/inference PASS or exact failure;
3. if it fails, run the provider-recommended multimodal Transformers load path under the same environment;
4. retain the distinction:
   • legacy path PASS -> current MVP request/result contract for first ui-grounding consumer #35 seam can preserve loader class while separating acquisition;
   • legacy path FAIL + recommended path PASS -> this is evidence for a separately bounded runtime-loader correction, not grounds to hide the problem inside MVP request/result contract for first ui-grounding consumer #35;
   • both FAIL -> external environment/model compatibility evidence; proceed to local target without manufacturing a pass.

Do not broaden into model modernization or benchmarking. The purpose is to cheaply falsify one loader compatibility assumption using someone else's GPU.

[mark-e-deyoung]

Web-lane execution claim — bounded external-smoke harness

I am taking only the public-safe harness portion of #39 in this web session. This does not replace or block the active first-consumer critical path.

Owned mutation surface:

• one portable external-GPU smoke script;
• one small public/offline contract test or CI check for that script;
• concise execution instructions/evidence schema.

The harness will pin the exact public KV-Ground revision, generate its own synthetic GUI fixture, compare the legacy-equivalent AutoModelForCausalLM load path against the provider evaluation path, and emit a machine-readable environment/result record. It will not import private desktop-ui-cv source, use WineBot frames, qualify target performance/quality, create a provider abstraction, or require external GPU availability.

Public GitHub Actions may be used only to validate the harness contract/syntax without downloading model weights. The actual GPU experiment remains opportunistic external/shared compute.

[mark-e-deyoung]

External-smoke harness ready — PR #40

The bounded public-safe harness is now a verified draft candidate:

• PR: Add bounded external KV-Ground compatibility smoke #40 Add bounded external KV-Ground compatibility smoke
• branch: experiment/issue-39-external-kv-ground-smoke
• exact verified head: 3ce1a352c9dcc056b1c0c1f31b7b0bf752ad2515
• public Actions run: 33592833988 — PASS
• base/control stack: docs/mvp-control-realignment / draft PR Realign Foundry agent contract and current task to identity-first MVP #22

The implementation pins the exact corrected-weight revision fe7563292bb52ab6c235fc3c87157e6a14017479, generates its own synthetic GUI fixture, and supports the required sequence:

1. legacy_equivalent → AutoModelForCausalLM;
2. only if needed, provider_eval → AutoModelForImageTextToText.

The receipt records public/synthetic environment/load/inference/output/parser evidence and no tracebacks/private data. A self-review correction before final verification also made structural parseability part of the PASS condition, so "generated some text" cannot satisfy the smoke.

The public CI deliberately exercised only the stdlib contract/parser/repeatability surface; it downloaded no model and requested no GPU.

Remaining #39 gate

Actual shared-GPU execution is still pending and remains opportunistic. I searched for a ready exact-model hosted demo but did not find one that answers this loader-class experiment without changing representation/runtime. Use a low-friction authorized CUDA environment when available; if adapting a free platform becomes its own project, stop #39 and continue the authoritative local target path as specified.