Skip to content

Drop Python 3.9/3.10 support, add Python 3.13#1111

Open
rkingsbury wants to merge 13 commits into
materialsproject:mainfrom
rkingsbury:drop-py39-310-add-py313-314
Open

Drop Python 3.9/3.10 support, add Python 3.13#1111
rkingsbury wants to merge 13 commits into
materialsproject:mainfrom
rkingsbury:drop-py39-310-add-py313-314

Conversation

@rkingsbury

Copy link
Copy Markdown
Collaborator
  • Updates requires-python to >=3.11
  • Updates classifiers to reflect supported versions (3.11-3.14)
  • Updates CI test matrix and upgrade-dependencies matrix to 3.11-3.14
  • Updates docs badge and text to say Python 3.11+
  • Removes requirements files for 3.9/3.10, adds placeholder requirements files for 3.13/3.14 (the upgrade-dependencies workflow will regenerate these with correct pinned versions)

NOTE: This PR was generated by Anthropic's Claude Code AI and reviewed by a human maintainer.

rkingsbury and others added 2 commits June 20, 2026 22:43
- Updates requires-python to >=3.11
- Updates classifiers to reflect supported versions (3.11-3.14)
- Updates CI test matrix and upgrade-dependencies matrix to 3.11-3.14
- Updates docs badge and text to say Python 3.11+
- Removes requirements files for 3.9/3.10, adds placeholder requirements
  files for 3.13/3.14 (the upgrade-dependencies workflow will regenerate
  these with correct pinned versions)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ruff auto-fixed 422 violations (UP007, UP035, UP038, B905, F401, I001,
UP017, PYI016); manually fix 6 that needed human intervention. Black
reformatted 3 files. All pre-commit hooks now pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rkingsbury rkingsbury closed this Jun 21, 2026
@rkingsbury rkingsbury reopened this Jun 21, 2026
@rkingsbury rkingsbury added pkg Package and repository health housekeeping labels Jun 21, 2026
@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.37383% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.53%. Comparing base (05d864c) to head (63c3e4e).

Files with missing lines Patch % Lines
src/maggma/api/query_operator/dynamic.py 0.00% 5 Missing ⚠️
src/maggma/api/resource/submission.py 0.00% 4 Missing ⚠️
src/maggma/api/resource/read_resource.py 0.00% 2 Missing ⚠️
src/maggma/api/resource/s3_url.py 0.00% 2 Missing ⚠️
src/maggma/api/API.py 0.00% 1 Missing ⚠️
src/maggma/api/query_operator/sorting.py 0.00% 1 Missing ⚠️
src/maggma/api/query_operator/sparse_fields.py 0.00% 1 Missing ⚠️
src/maggma/api/resource/aggregation.py 0.00% 1 Missing ⚠️
src/maggma/api/resource/post_resource.py 0.00% 1 Missing ⚠️
src/maggma/api/resource/utils.py 0.00% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1111      +/-   ##
==========================================
- Coverage   69.54%   69.53%   -0.02%     
==========================================
  Files          47       47              
  Lines        4118     4103      -15     
==========================================
- Hits         2864     2853      -11     
+ Misses       1254     1250       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

rkingsbury and others added 2 commits June 21, 2026 09:57
In Python 3.14, asyncio.get_event_loop() raises RuntimeError when called
outside a running event loop (rather than creating one with a
DeprecationWarning as in 3.10-3.13).

- cli/__init__.py: replace get_event_loop()/run_until_complete() with
  asyncio.run(), which is the correct modern pattern for driving an async
  function from synchronous code
- cli/multiprocessing.py: replace get_event_loop() with
  get_running_loop(), which is semantically precise (both call sites are
  inside running coroutines) and avoids the deprecated fallback path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Per Claude:
Most likely cause of the projection builder hang: The --cov=maggma flag in CI. Python 3.14 replaced sys.settrace-based coverage with the new sys.monitoring API (PEP 669), and older coverage.py versions (which the copied 3.14 requirements file pins) can deadlock on nested generator patterns. The projection builder's get_items() has a yield inside a loop that itself iterates a grouper generator — a pattern that is unique to the projection builder among the builder tests.
@rkingsbury rkingsbury closed this Jun 21, 2026
@rkingsbury rkingsbury reopened this Jun 21, 2026
rkingsbury and others added 3 commits June 21, 2026 10:05
Previously these were placeholder copies of the 3.12 requirements.
Generated using pip-compile 7.5.3 with pip 25.3 and --resolver=backtracking,
matching the logic in the upgrade-dependencies workflow.

Notable: coverage is now pinned to 7.14.2 (supports sys.monitoring, the
new tracing API in Python 3.14 that replaces sys.settrace-based coverage).

macos-latest variants require a macOS runner and should be generated by
triggering the upgrade-dependencies workflow after this PR merges.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove named `docs` generator variable from Projection_Builder.get_items()
  to avoid leaving an exhausted generator in the frame at yield time, which
  may interact with Python 3.14's sys.monitoring API used by coverage.py
- Replace datetime.utcnow() with datetime.now(UTC) throughout src/ and key
  test files (deprecated since 3.12, removed in future Python)
- Add pytest-timeout (2.4.0) to testing deps with --timeout=120 in addopts
  to convert any remaining hangs into failures with tracebacks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rkingsbury

Copy link
Copy Markdown
Collaborator Author

We observed that python 3.14 kept hanging on the projection builder test, motivating some additional modernization work summarized below (by Claude):

Root cause hypothesis: Python 3.14's sys.monitoring API (used by coverage.py for instrumentation) may have a bug or edge case when a generator frame is suspended at a yield point while a local variable still holds a reference to another (exhausted) generator. Projection_Builder.get_items() was uniquely susceptible because it kept docs = store.query(...) as a named local variable at the yield point. Every other builder (MapBuilder, GroupBuilder) either calls list(store.query(...)) or inlines the query directly in the for loop.

Fixes made:

  • [Primary fix] projection_builder.py: Changed docs = store.query(...); for d in docs: → for d in store.query(...): — eliminates the named generator variable from the frame at yield time, making it structurally identical to GroupBuilder's pattern.

  • [Cleanup + correctness] All datetime.utcnow() calls replaced with datetime.now(UTC) across 7 source files and test_copy_builder.py. Ruff further simplified these to use from datetime import UTC (UP017 rule), the preferred 3.11+ idiom.

  • [Diagnostic safety net] Added pytest-timeout==2.4.0 to testing deps and --timeout=120 to pytest addopts. If any test still hangs, it will now fail with a traceback showing exactly where it's stuck instead of keeping the CI job alive until the 7-minute job-level cancellation.

pymongo 4.10.1 predates Python 3.14 support and lacks a cp314 wheel.
mongomock depends on pymongo's bson C extension for every document
operation, so an incompatible bson build on Python 3.14 is the most
likely cause of the projection builder test hang. Removing the <4.11
upper bound allows pip to install pymongo 4.17.0 which includes
Python 3.14 wheel support.

All four ubuntu-latest requirements files regenerated with:
  pymongo==4.17.0
  dnspython==2.8.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rkingsbury rkingsbury closed this Jun 21, 2026
@rkingsbury rkingsbury reopened this Jun 21, 2026
CI hangs on 3.14 testing; revert to 3.13 as the max supported/tested
version until the hang is diagnosed. 3.11-3.13 compatibility fixes
made earlier in this branch are unaffected and remain in place.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rkingsbury rkingsbury closed this Jun 22, 2026
@rkingsbury rkingsbury reopened this Jun 22, 2026
rkingsbury and others added 3 commits June 21, 2026 23:08
The ubuntu extras lockfiles had drifted to listing specific extras
groups (--extra=api --extra=azure ...) instead of --all-extras, which
silently dropped docs/memray/mongogrant/notebook_runner deps from the
test matrix. Regenerate all ubuntu-latest_py3.{11,12,13} lockfiles
with pip-compile --all-extras, matching the upgrade-dependencies.yml
workflow command. Also fix a stale py3.12 filename left in the
macos-latest_py3.13 lockfile header comments after the 3.13 rename.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These were introduced to chase a suspected Python 3.14 hang in CI,
but 3.14 support has been dropped from this branch, so the bump is
no longer needed. Restore pymongo>=4.2.0,<4.11, the mongo:4.0 test
service image, and regenerate the ubuntu requirements lockfiles
accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rkingsbury rkingsbury closed this Jun 22, 2026
@rkingsbury rkingsbury reopened this Jun 22, 2026
@rkingsbury rkingsbury changed the title Drop Python 3.9/3.10 support, add Python 3.13/3.14 Drop Python 3.9/3.10 support, add Python 3.13 Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

housekeeping pkg Package and repository health

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant