[mykolamelnykml]

The backend test job took about 6 minutes, much longer than any other job. pytest accounted for 327s of it (707 tests).

Where the time went

A local --durations profile of the full suite (188s):

• Most of the time is benchmark tests. Each test that needs a scored submission spends about 3.3s in fixture setup, because finalize runs the real pdfredeval scorer inline (Q_CLUSTER is synchronous in tests).
• The scorer parity test takes 17s on its own.
• Without benchmark tests the suite runs in 14s locally.

What changes

• Marking (backend/conftest.py): a collection hook marks tests benchmark if they are in a test_benchmark* or test_mcp_benchmarks file, or use benchmark_case, case_files or run_files. That catches four tests in other files (API keys, CLI login, badges), and new tests are marked without anyone remembering to. Both markers are registered in pyproject.toml, since --strict-markers is on.

• Smoke test: test_an_uploaded_submission_is_scored_and_waits_for_review is marked benchmark_smoke. It covers the whole pipeline: case published, output uploaded, scored by the real scorer, waiting for review.

• CI runs pytest -m "not benchmark or benchmark_smoke" (557 of 707 tests). It no longer installs tesseract-ocr, which only the parity test needs; the smoke test passes with tesseract unavailable.

• New Benchmarks workflow (.github/workflows/benchmarks.yml) runs pytest -m benchmark (151 tests), with tesseract installed. It runs on PRs and pushes to main that touch:

  • backend/apps/benchmarks/**
  • the scorer submodule
  • the benchmark tests and fixtures
  • backend/conftest.py
  • the workflow file itself

  It can also be started by hand from the Actions tab. It is a separate workflow because paths filters a whole workflow, and because deploy.yml waits on "CI", which this must not hold up.

• make backend-test-fast runs what CI runs. make backend-test still runs everything.

• DEVELOPMENT.md and CLAUDE.md describe the split.

Trade-off

Coverage in the main CI run drops from 95% to 86%, almost all of it in apps/benchmarks/. The Benchmarks workflow covers that code when it changes. What isn't covered: a change outside those paths that breaks a benchmark test, for example a catalog model. The docs say to run make backend-test or start the workflow by hand in that case.

Testing

• make backend-test-fast: 557 passed, 150 deselected, in 14s (the full suite was 188s locally).
• pytest -m benchmark: 151 passed, in 2m57s.
• make backend-lint passes, and the workflow YAML parses with one shared path list for both triggers.
• This PR changes conftest.py and benchmarks.yml, so the Benchmarks workflow should run on it. That run is the check that the new workflow works on GitHub.