Skip to content

run-tests-uv: testpaths silently overrides root-dir — package doctests never run, CI stays green #56

Description

@thorwhalen

Summary

[tool.pytest.ini_options] testpaths silently overrides the root-dir input of the run-tests-uv action. In a repo with testpaths = ["tests"], the package's own doctests execute nowhere — not locally, not in CI — while CI reports green.

Why this is easy to miss

run-tests-uv invokes pytest with --doctest-modules but no path argument. Collection is therefore driven entirely by testpaths. The action's root-dir input only feeds --cov=, so it looks like the package dir is being collected when it is not. Coverage numbers can even look plausible because --cov still names the package.

Unlike ruff-ruleset drift, this failure mode does not turn a repo red. It keeps it green while testing nothing, so it is invisible until someone widens testpaths and watches the collected-item count jump.

Observed impact

Found in 3 of 8 repos in a single wave-0 rollout batch:

repo before after widening testpaths
i2mint/focal 1 item collected 7focal/tests/test_extension_store.py plus all module doctests had never run
i2mint/dol_cookbook pipes.py 0% covered 100%
i2mint/reci testpaths pointed at a tests/ dir that does not exist → silent pytest rootdir fallback

i2mint/doodad still has 3 latent doctest failures hidden this way (see i2mint/doodad), and i2mint/ij has 2 broken doctests plus one that hangs indefinitely (see i2mint/ij).

Proposed fix

Options, roughly in order of preference:

  1. Make run-tests-uv pass an explicit path (the package dir) rather than relying on testpaths, so the action's contract matches its documented behaviour.
  2. Have read-ci-config warn loudly when testpaths is set and excludes the package directory, so the repo owner sees it in the CI log.
  3. Add a wads audit check: "package contains >>> doctests but testpaths omits the package dir" → surface as a repo_audit finding.

⚠️ Hazard for any fleet-wide sweep

Do not naively widen testpaths across the fleet. At least one repo (i2mint/ij) has a package doctest that hangs indefinitelypytest <pkg> --doctest-modules never terminates. Any sweep needs a per-repo timeout and a way to quarantine hangers.

Reproduce

In an affected repo:

python -m pytest --collect-only -q                                  # what CI actually collects
python -m pytest <pkgdir> --doctest-modules --collect-only -q       # what you assumed it collects

Surfaced during the 2026-07-30 wave-0 rollout batch. Cross-ref: the running sweep journal in #51.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions