test: model consolidation and flexibility

[planetf1]

Context

Part of the Testing Infrastructure & Strategy Overhaul epic (item 3b).

Objective

Consolidate the models used across the test suite — catalog what's currently used, identify redundancy, and define a minimal default set per tier (local dev, CI, nightlies) to lower the barrier for contributors. At the same time, expand flexibility: implement override flags so developers and CI can substitute models easily, and evaluate additional models for qualitative/semantic tests.

Work

• Catalog all models currently used across the test suite and examples
• Identify redundancy and define a minimal default model set per tier (local dev, CI, nightlies)
• Implement override flags (env vars / pytest flags) so developers and CI can substitute models easily
• Evaluate Qwen as an additional model for qualitative/semantic tests
• Address Migrate HuggingFace Tests to Granite 4 When Adapters Available #359 (migrate HF tests to Granite 4 when adapters available)
• Ensure Test model_ids.py for missing models, and add graceful handling when models don't exist #297 (test model_ids.py for missing models) is addressed across both this issue and test: environment diagnostic, pre-flight checks & execution reporting #731
• Update contributor documentation on model requirements and overrides

Notes

• This covers both test/ and example scripts
• Migrate HuggingFace Tests to Granite 4 When Adapters Available #359 is a sub-issue of this work item
• Test model_ids.py for missing models, and add graceful handling when models don't exist #297 spans both this issue and test: environment diagnostic, pre-flight checks & execution reporting #731 (environment diagnostics) — referenced in both but parented under test: environment diagnostic, pre-flight checks & execution reporting #731

[ajbozarth]

Found a potentially related issue and Claude suggested posting it in this issue instead of a new one:

---

While reviewing PR #818 I hit failures in two examples that were previously
silently skipped due to the requires_heavy_ram marker (48 GB RAM threshold).
That marker was removed in #742, and the directory traversal fix in #795 means
these examples now collect and run unconditionally — exposing that they require
models most contributors won't have pulled:

• docs/examples/safety/repair_with_guardian.py → llama3.2 (untagged)
• docs/examples/image_text_models/vision_openai_examples.py → qwen2.5vl:7b

Both are documented in CONTRIBUTING.md's pull script, but there's no runtime
enforcement — the ollama marker only checks that Ollama is running, not that
a specific model exists. The result is an opaque 404 crash rather than a
meaningful skip/xfail.

Proposed addition to test/predicates.py: a require_ollama_model(name)
predicate that queries /api/tags at collection time and marks the test as
xfail (not skip) when the model isn't present. xfail is preferable here
because the test should pass given the right setup — a skip would hide the
gap from CI and from contributors who want to know what they're missing.

This fits naturally into the "minimal default model set per tier" and
"contributor barrier" goals of this issue. The predicate pattern already exists
for GPU, RAM, API keys, and packages — model availability is a natural addition
to that set.

Example usage in the affected files:

pytestmark = [pytest.mark.e2e, pytest.mark.ollama, require_ollama_model("llama3.2")]

Happy to open a focused PR for the predicate if that's useful, or fold it into
the broader model consolidation work here.

---