Skip to content

[Testing] Migrate accelerator-backed tests to vLLM CPU servers in CI #748

Description

@Bslabe123

The tests that needed accelerators never ran in CI, and most of what they check does not depend on GPU speed, so they move to real vLLM CPU servers inside the existing E2E Test on change job. This is the #703 pattern (a CPU vLLM as the real-server oracle, implemented for token accounting and /metrics in #697) applied to the rest of the accelerator-backed surface.

What was accelerator-backed

tests/optional, the hand-run @pytest.mark.live tier: eight multimodal cases (images, video, mixed, frame sampling, two shared-prefix variants, VisionArena, audio) and one text chat case, each deploying Qwen3-VL-8B or Qwen2-Audio-7B on an H100 node through deploy/manifests.yaml and asserting that every request completes. With no accelerator capacity in CI (#641), the tier ran only when someone ran it by hand, so changes to multimodal payload building shipped with no real-server check.

The rule for what moves

Fake the conditions, never the oracle (#606): a case moves when its oracle is the server accepting a payload, or the server's own numbers, because neither depends on model size or speed, and the sim cannot stand in (it matches multimodal blocks by type string and never decodes bytes). A case stays on a cluster only when the cluster is the oracle.

Work

  • test: run the optional tier's payload cases against vLLM CPU servers #745: the nine payload cases become e2e/tests/test_vllm_cpu_multimodal.py against OpenGVLab/InternVL3-1B-hf (image + video) and ibm-granite/granite-4.0-1b-speech (audio), run by a multimodal-e2e job on ARM runners per release in e2e/vllm_multimodal_releases.txt, merge-blocking from its first run. tests/optional keeps text/chat only.

Stays accelerator-backed, on purpose

Part of the v0.7.0 release (#606), Test infrastructure and gating.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions