Skip to content

Single-source the version, and stop the local test run lying - #34

Merged
benjipeng merged 1 commit into
mainfrom
chore/single-source-version-and-honest-skips
Aug 7, 2026
Merged

Single-source the version, and stop the local test run lying#34
benjipeng merged 1 commit into
mainfrom
chore/single-source-version-and-honest-skips

Conversation

@benjipeng

Copy link
Copy Markdown
Contributor

Three fixes to the build and test surface, all found by actually running the commands the README tells a developer to run.

The version is read from one place

pyproject.toml declared a literal alongside the one in __init__.py. Nothing failed loudly when they disagreed — a wheel would just misreport itself. It is now dynamic = ["version"] with [tool.hatch.version] pointing at __init__.py.

Proved genuinely dynamic rather than coincidental: setting __version__ to 9.9.9 builds mlx_cv-9.9.9-py3-none-any.whl. Restored immediately; git diff on __init__.py is empty.

pytest from the repository root no longer detonates

There were no testpaths, so collection walked into references/, where mlx-vlm's smoke test calls sys.exit(1) at import time and takes the whole run down with an INTERNALERROR.

CI never saw this because references/ is untracked — so the breakage was invisible to the one place that runs the suite automatically, and unavoidable on every developer machine that has the upstream checkouts.

Two heavy gates reported environment gaps as parity failures

Both checked that the external assets were present but not that they could be read:

  • the SAM 3.1 contract needs PyTorch
  • the DA3 video case needs OpenCV

With the weights on disk and neither package installed, they failed rather than skipped — red that says "parity broke" when nothing had been compared at all. That is the worst kind of red, because it trains people to ignore it.

They skip with the reason now. DA3's required mode still fails, so a gate that is supposed to run cannot vanish quietly. This follows the pattern the DA3 suite already used elsewhere (_or_skip helpers, pytest.importorskip("cv2")) — the failing paths had simply missed it.

Result

before after
bare pytest INTERNALERROR 454 passed
with testpaths only 2 failed, 454 passed
final 454 passed, 17 skipped, 0 failed

Both gates verified to emit a reason:

SKIPPED tests/test_sam31_reference_contract.py:164: official SAM 3.1 checkpoint inspection requires PyTorch
SKIPPED tests/test_da3_upstream_parity.py:316: DA3 video frame loading requires OpenCV/cv2

Three fixes to the build and test surface, all found by actually running
the commands the README tells a developer to run.

**The version is read from one place.** pyproject.toml declared a literal
alongside the one in __init__.py. Nothing failed loudly when they
disagreed — a wheel would just misreport itself. It is now
`dynamic = ["version"]` with `[tool.hatch.version]` pointing at
__init__.py. Proved dynamic rather than coincidental: setting __version__
to 9.9.9 builds mlx_cv-9.9.9-py3-none-any.whl.

**`pytest` from the repository root no longer detonates.** There were no
testpaths, so collection walked into references/, where mlx-vlm's smoke
test calls sys.exit(1) at import time and takes the run down with an
INTERNALERROR. CI never saw this because references/ is untracked, so the
breakage was invisible to the one place that runs the suite automatically
and unavoidable on every developer machine that has the checkouts.

**Two heavy gates reported environment gaps as parity failures.** Both
checked that the external assets were present but not that they could be
read: the SAM 3.1 contract needs PyTorch, the DA3 video case needs
OpenCV. With the weights on disk and neither package installed, they
failed rather than skipped — red that says "parity broke" when nothing had
been compared at all. They skip with the reason now. DA3's required mode
still fails, so a gate that is supposed to run cannot vanish quietly.

Before: INTERNALERROR, then 2 failed / 454 passed.
After: 454 passed, 17 skipped, 0 failed.
@benjipeng
benjipeng merged commit 9fc066f into main Aug 7, 2026
2 checks passed
@benjipeng
benjipeng deleted the chore/single-source-version-and-honest-skips branch August 7, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant