Skip to content

Check for regressions between PyPI and reqstool-client@main CI runs #105

Description

@jimisola

Context

`.github/workflows/build.yml` now runs the `reqstool status`/`reqstool validate` steps twice via a matrix (`reqstool-source: [pypi, main]`) — once against the latest released PyPI `reqstool` package, once against `reqstool-client`'s `main` branch (installed via `pip install "reqstool @ git+https://github.com/reqstool/reqstool-client.git@main"\`).

This was added so CI exercises both the stable release and the in-development `main` branch, since `reqstool-client` is intentionally holding off on its next release until the org-wide OpenSpec dogfooding rollout (see `PLAN_dog_fooding.md`) is complete — `main` has accumulated real fixes (e.g. #411) that aren't on PyPI yet.

What to check

  • Periodically compare the two matrix legs' `reqstool status` output for unexpected divergence (beyond the known, intentional gap — `reqstool validate --strict` only runs on the `main` leg since that subcommand isn't published yet).
  • Once `reqstool-client` cuts its next release, confirm the `pypi` leg's output matches the `main` leg's output exactly, then collapse this back down to a single non-matrixed job (drop the `main` leg, restore `reqstool validate --strict` unconditionally).
  • Watch for the `pypi` leg silently masking a regression that only the `main` leg would catch (or vice versa) during the rollout period.

Related: reqstool-client#412 (CLI/MCP consolidation, found during this same rollout).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions