Skip to content

fix: CI regression — Run CI doubled from ~17 min to ~35 min after PR #742 #903

Description

@planetf1

Observed behaviour

The Run CI workflow duration jumped from ~17–20 minutes to ~35–40 minutes, first seen on 2026-03-31. The increase is consistent across all three Python matrix jobs.

Root cause

PR #742 (#742) accidentally removed @pytest.mark.block_network from test_openai_compat in test/formatters/granite/test_intrinsics_formatters.py during the marker taxonomy audit.

That test deliberately connects to an unreachable URL (http://example.com:98765/...) and expects an openai.APIConnectionError. The comment in the test still reads "Note that network access is blocked for this test case."

With the marker in place, pytest-recording intercepts the socket call immediately. Without it, the OpenAI client makes a real TCP connection and waits for the OS-level connect timeout — approximately 31 seconds per call. The test makes 2 calls per parameterised case across 17 cases:

17 cases × 2 calls × ~31 s = ~18 minutes extra per matrix job

Evidence

Run Tests Passed Skipped
2026-03-30 (pre-regression) 711 s 721 151
2026-04-07 (post-regression) 1666 s 873 125

Pre-regression each test_openai_compat case completed in ~2.8 s; post-regression each takes ~63 s.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions