Skip to content

ci: add regression tests for the version-bump and docs workflows #427

Open

Description

@saucer-boy

What

Add automated tests to the repository that keep the job split of version-bump.yml and docs.yml working, so that a later edit cannot break it unnoticed.

Why

In both workflows, the jobs that hold a write token (push in version-bump.yml, deploy in docs.yml) run only git and shell. They check everything that the read-only jobs hand them before they push. Those checks were tested with local test harnesses that are not part of the repository. Without tests in CI, a later edit to a workflow, a change to the bump configuration or a tool update could break a check, or the normal release path, and the first sign would be a refused release.

Scope

  • A pytest module, run in the existing "Test uv" jobs on Ubuntu and macOS (skipped on Windows). It takes the run: text of the push and deploy steps from the workflow files and runs it with bash and git against inputs built with git plumbing:
    • bundles with extra refs, extra commits, changed or extra files, or forged commit and tag headers;
    • free text in the version commit or tag message;
    • half-done bumps;
    • lock files changed in any TOML spelling;
    • sites with symlinks, control characters, .git paths or a changed CNAME.
  • The normal path: a real version bump made on a copy of the repository is accepted.
  • A property test of the lock rule, with Python's TOML parser as an independent check.
  • actionlint, with shellcheck, on the workflow files in CI.
  • A check that no package tool (uv, uvx, python, mkdocs, bump-my-version) appears in a job that holds a write token.

Done when

  • Removing any single check from push or deploy makes at least one test fail.
  • The tests pass on main.
  • They add no more than a few minutes to the test jobs.

Refs #418

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciContinuous IntegrationenhancementNew feature or requestsecurity

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions