fix[ci]: modernize CI and petl dependencies by juarezr · Pull Request #702 · petl-developers/petl

juarezr · 2026-07-20T18:53:02Z

This PR has the objective of fixing CI issues and upgrading python depencies.

Changes

Added notice/warning messages on optional behaviors on the CI
Fixed PETL build moving away from sdist and wheels
Fixed issues with optional dependencies: bcolz and Cython
Fixed issues with optional dependencies: numpy and tables
Added bcolz-zipline to test on CI instead of bcolz for python v3.9+
Fix the CodeQL CI workflow by upgrading to python v3.10
Upgrade GitHub Actions version
Move coverage testing from python v3.8 to python v3.10

Checklist

Use this checklist to ensure the quality of pull requests that include new code and/or make changes to existing code.

qodo-free-for-open-source-projects · 2026-07-20T18:53:49Z

PR Summary by Qodo

CI: switch workflow builds from setup.py to PEP 517 (python -m build)

🐞 Bug fix ⚙️ Configuration changes 🕐 10-20 Minutes

AI Description

• Replace setup.py-based builds in CI with PEP 517 builds via python -m build.
• Install/upgrade build tooling in workflows to avoid CI packaging failures.

Diagram

graph TD
  A["GitHub Actions"] --> B["Test job"] --> C["Build petl (PEP 517)"] --> D["Run pytest"]
  A --> E["Docs job"] --> F["Build petl (PEP 517)"] --> G["Run sphinx"]

High-Level Assessment

The following are alternative approaches to this PR:

1. Install directly with PEP 517 (pip install .)

➕ Simpler than producing dist artifacts; mirrors typical user installs
➕ Avoids needing python -m build output management
➖ Does not explicitly validate sdist/wheel buildability
➖ May hide packaging regressions that only appear during build

2. Use `pipx run build` for isolated build tooling

➕ Avoids polluting the job environment with upgraded build dependencies
➕ More reproducible tooling versions per invocation
➖ Adds another tool (pipx) and moving parts to CI
➖ Less common pattern for Python library CI compared to python -m pip install build

Recommendation: Keep the PR’s approach: using python -m build is the most direct modernization away from setup.py sdist/bdist_wheel while explicitly validating packaging. The only follow-up worth considering is pinning/controlling build tool versions if CI stability becomes an issue.

Files changed (1) +6 / -2

Other (1) +6 / -2

test-changes.ymlModernize CI build step to 'python -m build' +6/-2

Modernize CI build step to 'python -m build'

• Replaces legacy 'setup.py sdist bdist_wheel' and 'setup.py build' steps with a PEP 517 build ('python -m build'). Ensures 'build', 'setuptools', and 'wheel' are upgraded/available before building in both the main test matrix job and the documentation build job.

.github/workflows/test-changes.yml

coveralls · 2026-07-20T18:57:09Z

Coverage Report for CI Build 29780656074

Warning

No base build found for commit 09104ec on master.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 91.581%

Details

Patch coverage: 6 uncovered changes across 3 files (54 of 60 lines covered, 90.0%).

Uncovered Changes

File	Changed	Covered	%
petl/test/io/test_xls.py	5	3	60.0%
petl/test/io/test_xlsx.py	50	48	96.0%
petl/test/io/test_xml.py	5	3	60.0%

Coverage Regressions

Requires a base build to compare against. How to fix this →

Coverage Stats


Relevant Lines:	14955
Covered Lines:	13696
Line Coverage:	91.58%
Coverage Strength:	0.92 hits per line

💛 - Coveralls

qodo-free-for-open-source-projects · 2026-07-20T18:58:28Z

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Build step breaks py27 🐞 Bug ☼ Reliability

Description

The workflow now unconditionally installs and runs the build frontend (python -m build) in the
main test matrix, which includes a Python 2.7 entry. This introduces a new packaging-tool dependency
into the Python 2.7 job and can fail CI before tests run if that tool cannot be installed/executed
on that interpreter.

Code

.github/workflows/test-changes.yml[R196-199]

- name: Setup petl package with mode = ${{ matrix.mode }}
-        run: python setup.py sdist bdist_wheel
+        run: |
+          python -m pip install --upgrade build setuptools wheel
+          python -m build

Evidence
The workflow’s matrix includes a Python 2.7 job, and the modified step installs/runs `python -m
build` without any condition, so it will execute under Python 2.7 as well.
.github/workflows/test-changes.yml[38-75]
.github/workflows/test-changes.yml[196-200]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The CI matrix includes Python 2.7, but the workflow now always installs the `build` frontend and runs `python -m build`. This adds a new dependency path for the py2.7 job and can prevent that job from ever reaching pytest.
### Issue Context
- The matrix explicitly includes `python: "2.7"`.
- The build step has no `if:` guard and will execute for that entry.
### Fix Focus Areas
- .github/workflows/test-changes.yml[38-75]
- .github/workflows/test-changes.yml[196-200]
### Suggested fix
- Add an `if:` guard so the `python -m pip install ... build` + `python -m build` step runs only for Python >= 3 (or `matrix.python != '2.7'`).
- For the Python 2.7 leg, either:
- keep the legacy command (`python setup.py sdist bdist_wheel`), or
- skip building entirely if it’s not needed for the test run.
- Optionally, pin build tool versions (or use a constraints file) to reduce future CI breakage.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. NumPy/PyTables marker gap 🐞 Bug ☼ Reliability

Description

requirements-formats.txt forces NumPy >2.0.0 for Python >3.9 but only constrains PyTables to
>=3.9.2, which still permits selecting tables==3.9.2 alongside NumPy 2.x despite the file’s own
compatibility comments. This can yield an unusable environment (install/import/test failures) in
full-scope CI jobs that install requirements-formats.txt.

Code

requirements-formats.txt[R1-6]

+Cython                 ; python_version < '3.7' or python_version >= '3.9'
+Cython>=0.29.21,<3.0.0 ; python_version >= '3.7' and python_version < '3.9' # fixes further bcolz build
+
+numpy<2.0.0 ; python_version <= '3.9' # tables==3.9.2 fails with numpy 2.x
+numpy>2.0.0 ; python_version >  '3.9' # tables>=3.10 requires numpy>2.0.0
+

Evidence

The requirements file explicitly forces NumPy 2.x for Python >3.9 while simultaneously allowing
PyTables 3.9.2+, and its own comments call out that tables==3.9.2 fails with NumPy 2.x and that
tables>=3.10 pairs with NumPy>2. The CI workflow installs requirements-formats.txt in full-scope
jobs, so this marker gap can surface as dependency/install/runtime breakage depending on which
tables build is selected.

requirements-formats.txt[1-6]
requirements-formats.txt[21-22]
.github/workflows/test-changes.yml[198-203]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`requirements-formats.txt` currently mandates `numpy>2.0.0` for `python_version > '3.9'` but does not also require a `tables` version that is compatible with NumPy 2.x. The same file comments indicate `tables==3.9.2` fails with NumPy 2.x and that `tables>=3.10` requires NumPy 2.x, so the markers should be aligned to prevent an internally-documented incompatible combination.
### Issue Context
This matters because CI installs `-r requirements-formats.txt` for full-scope jobs, so any resolver/platform combination that selects an older `tables` (still allowed by `tables>=3.9.2`) while NumPy is forced to 2.x can break the job.
### Fix Focus Areas
- requirements-formats.txt[1-6]
- requirements-formats.txt[21-22]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. Tests not run from wheel 🐞 Bug ⚙ Maintainability

Description

CI runs pytest and Sphinx against the repository checkout (not an installed wheel/editable install)
even though petl/version.py is a generated (gitignored) file required for import petl. This
makes the test environment depend on packaging side effects and does not verify that the built
artifact is importable/behaves correctly when installed.

Code

.github/workflows/test-changes.yml[R196-199]

- name: Setup petl package with mode = ${{ matrix.mode }}
-        run: python setup.py sdist bdist_wheel
+        run: |
+          python -m pip install --upgrade build setuptools wheel
+          python -m build

Evidence
The codebase requires a generated petl/version.py for imports, and the workflow runs pytest/Sphinx
without installing petl, meaning the environment depends on generated artifacts rather than
validating an installation.
petl/init.py[1-6]
.gitignore[176-178]
setup.py[38-42]
.github/workflows/test-changes.yml[196-237]
.github/workflows/test-changes.yml[276-284]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow builds the package but then runs pytest and docs generation against the source tree, which relies on generated files (e.g., `petl/version.py`) and doesn’t validate the installed artifact.
### Issue Context
- `petl/__init__.py` imports `petl.version`, while `petl/version.py` is gitignored and generated via setuptools-scm.
- The workflow does not `pip install .` / `pip install -e .` / `pip install dist/*.whl` before running pytest and Sphinx.
### Fix Focus Areas
- .github/workflows/test-changes.yml[196-237]
- .github/workflows/test-changes.yml[276-284]
- petl/__init__.py[1-6]
- setup.py[38-42]
- .gitignore[176-178]
### Suggested fix
After `python -m build`, install the built wheel and run tests/docs against that installed distribution, e.g.:
- `python -m pip install dist/*.whl`
- then run `pytest ...`
For docs, either install the wheel first or switch to `pip install -e .` (editable) so imports don’t depend on whether a generated file was left in the checkout.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-free-for-open-source-projects · 2026-07-20T18:58:29Z

      - name: Setup petl package with mode = ${{ matrix.mode }}
-        run: python setup.py sdist bdist_wheel
+        run: |
+          python -m pip install --upgrade build setuptools wheel
+          python -m build


1. Build step breaks py27 🐞 Bug ☼ Reliability

The workflow now unconditionally installs and runs the build frontend (python -m build) in the main test matrix, which includes a Python 2.7 entry. This introduces a new packaging-tool dependency into the Python 2.7 job and can fail CI before tests run if that tool cannot be installed/executed on that interpreter.

Agent Prompt

### Issue description The CI matrix includes Python 2.7, but the workflow now always installs the `build` frontend and runs `python -m build`. This adds a new dependency path for the py2.7 job and can prevent that job from ever reaching pytest. ### Issue Context - The matrix explicitly includes `python: "2.7"`. - The build step has no `if:` guard and will execute for that entry. ### Fix Focus Areas - .github/workflows/test-changes.yml[38-75] - .github/workflows/test-changes.yml[196-200] ### Suggested fix - Add an `if:` guard so the `python -m pip install ... build` + `python -m build` step runs only for Python >= 3 (or `matrix.python != '2.7'`). - For the Python 2.7 leg, either: - keep the legacy command (`python setup.py sdist bdist_wheel`), or - skip building entirely if it’s not needed for the test run. - Optionally, pin build tool versions (or use a constraints file) to reduce future CI breakage.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-free-for-open-source-projects · 2026-07-20T18:58:29Z

      - name: Setup petl package with mode = ${{ matrix.mode }}
-        run: python setup.py sdist bdist_wheel
+        run: |
+          python -m pip install --upgrade build setuptools wheel
+          python -m build


2. Tests not run from wheel 🐞 Bug ⚙ Maintainability

CI runs pytest and Sphinx against the repository checkout (not an installed wheel/editable install) even though petl/version.py is a generated (gitignored) file required for import petl. This makes the test environment depend on packaging side effects and does not verify that the built artifact is importable/behaves correctly when installed.

Agent Prompt

### Issue description The workflow builds the package but then runs pytest and docs generation against the source tree, which relies on generated files (e.g., `petl/version.py`) and doesn’t validate the installed artifact. ### Issue Context - `petl/__init__.py` imports `petl.version`, while `petl/version.py` is gitignored and generated via setuptools-scm. - The workflow does not `pip install .` / `pip install -e .` / `pip install dist/*.whl` before running pytest and Sphinx. ### Fix Focus Areas - .github/workflows/test-changes.yml[196-237] - .github/workflows/test-changes.yml[276-284] - petl/__init__.py[1-6] - setup.py[38-42] - .gitignore[176-178] ### Suggested fix After `python -m build`, install the built wheel and run tests/docs against that installed distribution, e.g.: - `python -m pip install dist/*.whl` - then run `pytest ...` For docs, either install the wheel first or switch to `pip install -e .` (editable) so imports don’t depend on whether a generated file was left in the checkout.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

…agile

juarezr · 2026-07-20T21:40:59Z

/review -i

qodo-free-for-open-source-projects · 2026-07-20T21:45:56Z

+Cython                 ; python_version < '3.7' or python_version >= '3.9'
+Cython>=0.29.21,<3.0.0 ; python_version >= '3.7' and python_version < '3.9' # fixes further bcolz build
+
+numpy<2.0.0 ; python_version <= '3.9' # tables==3.9.2 fails with numpy 2.x
+numpy>2.0.0 ; python_version >  '3.9' # tables>=3.10 requires numpy>2.0.0
+


1. Numpy/pytables marker gap 🐞 Bug ☼ Reliability

requirements-formats.txt forces NumPy >2.0.0 for Python >3.9 but only constrains PyTables to >=3.9.2, which still permits selecting tables==3.9.2 alongside NumPy 2.x despite the file’s own compatibility comments. This can yield an unusable environment (install/import/test failures) in full-scope CI jobs that install requirements-formats.txt.

Agent Prompt

### Issue description `requirements-formats.txt` currently mandates `numpy>2.0.0` for `python_version > '3.9'` but does not also require a `tables` version that is compatible with NumPy 2.x. The same file comments indicate `tables==3.9.2` fails with NumPy 2.x and that `tables>=3.10` requires NumPy 2.x, so the markers should be aligned to prevent an internally-documented incompatible combination. ### Issue Context This matters because CI installs `-r requirements-formats.txt` for full-scope jobs, so any resolver/platform combination that selects an older `tables` (still allowed by `tables>=3.9.2`) while NumPy is forced to 2.x can break the job. ### Fix Focus Areas - requirements-formats.txt[1-6] - requirements-formats.txt[21-22]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-free-for-open-source-projects · 2026-07-20T21:45:57Z

Code review by qodo was updated up to the latest commit 8156d64

fix[ci]: modernize build petl awasy from sdist and wheels

66d8933

qodo-free-for-open-source-projects Bot reviewed Jul 20, 2026

View reviewed changes

juarezr added 17 commits July 20, 2026 16:14

ci: install petl package after build and before testing

00a951a

ci: fixed issues with optional dependencies: bcolz and Cython

a43c701

build: Fixed issues with optional dependencies: numpy and tables

8aa5ddc

ci: added some notice messages to the testing workflow

00b10cc

ci: upgrade to actions/checkout@v7 and actions/setup-python@v7

d4819ab

ci: upgrade CodeQL to run on python 3.10

7880a33

tests: replace imports of pkg_resources by importlib.resources.files

40d3528

ci: brew install hdf5

b72fdfb

ci: fix importlib.resources for python up to 3.8

d543eab

ci: skip testing tables on macos-latest because building on arm is fr…

8149a1d

…agile

ci: fix python2.7 unicode issue on xls testing

55c2bfa

ci: test bcolz-zipline for python 3.9 to 3.12

905c8b6

ci: fix ambiguity on requiment imports

f22ce4e

ci: fix importlib_resources on python version up to 3.9

b4725ed

ci: fix python2.7 unicode issues

d0fd5a6

ci: set python 3.10 as primary version for code coverage and docs

b085cf7

chore: update changes in docs for version = 1.7.20

8156d64

qodo-free-for-open-source-projects Bot reviewed Jul 20, 2026

View reviewed changes

juarezr merged commit 97fbbfd into petl-developers:master Jul 20, 2026
29 checks passed

juarezr deleted the fix/ci-petl-setup branch July 20, 2026 21:53

Navigation Menu

Uh oh!

fix[ci]: modernize CI and petl dependencies - #702

fix[ci]: modernize CI and petl dependencies#702
juarezr merged 18 commits into
petl-developers:masterfrom
juarezr:fix/ci-petl-setup

juarezr commented Jul 20, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026

Uh oh!

coveralls commented Jul 20, 2026 •

edited

Loading

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Uh oh!

juarezr commented Jul 20, 2026

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

juarezr commented Jul 20, 2026

Changes

Checklist

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026

PR Summary by Qodo

Uh oh!

coveralls commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 29780656074

Coverage: 91.581%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

juarezr commented Jul 20, 2026

Uh oh!

qodo-free-for-open-source-projects Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-free-for-open-source-projects Bot commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coveralls commented Jul 20, 2026 •

edited

Loading

qodo-free-for-open-source-projects Bot commented Jul 20, 2026 •

edited

Loading