Skip to content

Make all four backends safe under free-threaded CPython - #72

Merged
mosquito merged 8 commits into
masterfrom
free-threading
Aug 4, 2026
Merged

Make all four backends safe under free-threaded CPython#72
mosquito merged 8 commits into
masterfrom
free-threading

Conversation

@mosquito

@mosquito mosquito commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes #64.

The scope narrowed as real bugs turned up while testing under a genuinely free-threaded interpreter (PYTHON_GIL=0, forcing the GIL off despite none of the C extensions declaring Py_mod_gil support yet):

  1. Cross-Context claim race (all four backends). submit() (and linux_uring's cancel()) claimed an Operation via a plain check-then-set of in_progress - safe only because the GIL serialized it. Two Contexts racing to submit() the same Operation could both see it unclaimed and both dispatch it. Reproduced as a stable segfault in thread_aio (two workers running the same Operation concurrently), silent double-delivery in linux_aio/linux_uring, and a double-submit in python_aio. Fixed with a CAS on in_progress in the three C backends, and a real per-Operation lock in python_aio (the Context's own lock only serializes submissions through that Context).

  2. linux_uring ring bookkeeping race. submit()/cancel()/uring_drain_cq() each read the ring's head/tail once, do a batch of work sized off that snapshot, and only commit a new head/tail at the end. Two concurrent callers (typically concurrent process_events() drainers) could claim the same CQE range and each fire every callback in it - occasionally corrupting the ring enough to livelock the whole process. Fixed with a critical section (Py_BEGIN_CRITICAL_SECTION, compiles to nothing under the GIL) around each ring-bookkeeping section, always released before invoking a Python callback so a reentrant call can't deadlock on itself.

  3. Callback field race (linux_aio/linux_uring). op->callback wasn't safe to read concurrently with set_callback() replacing it - both now go through the same per-object critical section instead of a bare pointer read/write.

Tests

tests/test_free_threading.py: a GIL-stays-disabled regression check, plus stress tests (import-time GIL check, concurrent submit/claim races, concurrent process_events() drainers, concurrent result readers) across every backend, all gated on actually running under a free-threaded interpreter with the GIL disabled.

CI / releases

  • CI's 3.13t/3.14t jobs now set PYTHON_GIL=0 so they exercise real no-GIL concurrency for every backend, not just build-and-import. A separate free-threaded-python-only job proves python_aio alone stays genuinely no-GIL with no override, since it's the only backend importable there today (SOABI mismatch).
  • The release workflow, Makefile, and make-wheels.sh build cp313t/cp314t wheels alongside existing versions on every platform that builds C extensions (Windows ships pure-Python only, already version-agnostic). macOS's wheel job switched from actions/setup-python to uv for interpreter provisioning - the one already confirmed locally to handle free-threaded version strings correctly.

Caveat: none of the three C extensions declare Py_mod_gil yet, so a real end-user install on a free-threaded interpreter still gets the GIL silently re-enabled on import unless they set PYTHON_GIL=0 themselves. Declaring it requires converting each extension's module init to multi-phase - a bigger follow-up once this lands and gets some real-world mileage.

Benchmark: GIL vs free-threaded

Throughput vs concurrency, 16 KB sequential, all four backends, PYTHON_GIL=0 forcing the bottom row genuinely off:

Throughput vs concurrency, GIL vs free-threaded

linux_aio/linux_uring hold up close to the GIL baseline (the CAS/critical-section fixes cost little); thread_aio and python_aio show more volatility at high concurrency without the GIL's implicit serialization smoothing things out - expected for pool-based backends now doing real concurrent work instead of GIL-interleaved work.

Test plan

  • ruff/mypy clean
  • Full suite green on macOS (thread_aio/python_aio) and the Linux VM (all four backends)
  • Full suite green under a genuinely free-threaded interpreter with PYTHON_GIL=0, all four backends built fresh against it
  • thread_aio segfault: reproduced reliably pre-fix (5/20 runs), 0/20 post-fix
  • python_aio cross-Context race: reproduced (117/20000 double-submits) pre-fix, 0 post-fix, stable across repeated runs
  • linux_uring livelock: reproduced via gdb/py-spy-style live inspection pre-fix, full 144-case concurrent process_events() matrix passes post-fix with no hangs

None of the three C extensions declare Py_mod_gil support, so importing
any of them under a free-threaded interpreter silently re-enables the
GIL for the whole process - not attempting that here.

Two CI checks: a dedicated free-threaded-python-only job builds
normally under a regular interpreter, then runs the suite through the
matching free-threaded one against that same install - the compiled
extensions' SOABI doesn't match, so they fail to import exactly like
on any unsupported platform, leaving python_aio genuinely running with
no GIL. Separately, the existing tests matrix gets 3.13t/3.14t entries
that build fresh under a free-threaded interpreter directly, to see
what happens when the C extensions do compile there (they do; the GIL
just comes back).

Adds a regression test asserting the GIL actually stays off whenever
python_aio is the only backend available, plus a concurrency stress
test hammering python_aio's own locking with many concurrent
read/write ops across a real multi-worker ThreadPool.
Context._execute() serialized the check-and-set of operation.in_progress
under its own lock, which only protects submissions through that one
Context - two different Contexts submitting the same Operation shared
no lock and could both observe it unclaimed. Reproduced reliably under
a free-threaded interpreter (~0.5% of submissions double-dispatched).

Moves the claim itself onto the Operation via its own lock
(_claim()/_unclaim()), independent of which Context is dispatching it.
Fixes #64

Every backend's submit() (and linux_uring's cancel()) claimed an
Operation via a plain check-then-set of in_progress - safe only because
the GIL serialized it. Under a free-threaded interpreter, two Contexts
racing to submit() the same Operation could both see it unclaimed and
both dispatch it: reproduced as a stable segfault in thread_aio (two
workers running the same Operation concurrently), silent double-
delivery in linux_aio/linux_uring, and the same double-submit in
python_aio. Fixed with a CAS on in_progress in the three C backends,
and a proper per-Operation lock (instead of the Context's own, which
only serializes submissions through that one Context) in python_aio.

linux_uring's own submit()/cancel()/uring_drain_cq() had a second,
separate race: each reads the ring's head/tail once, does a batch of
work sized off that snapshot, and only commits a new head/tail at the
end. Two concurrent callers (typically concurrent process_events()
drainers) could claim the same CQE range and each fire every callback
in it - occasionally corrupting the ring enough to livelock the whole
process. Fixed with a critical section (Py_BEGIN_CRITICAL_SECTION,
compiles to nothing under the GIL) around each ring bookkeeping
section, always released before invoking a Python callback so a
reentrant call from inside one can't deadlock on itself.

linux_aio/linux_uring's callback field also wasn't safe to read
concurrently with set_callback() replacing it - both now go through
the same per-object critical section instead of a bare pointer
read/write, with the old callback properly released after.

Adds tests/test_free_threading.py: a GIL-stays-disabled regression
check, and stress tests (import-time GIL check, concurrent
submit/claim races, concurrent process_events() drainers, concurrent
result readers) across every backend, gated on actually running under
a free-threaded interpreter with the GIL disabled.

CI gets two free-threaded ubuntu jobs (3.13t/3.14t) that force
PYTHON_GIL=0 so the suite exercises real no-GIL concurrency, plus a
free-threaded-python-only job proving python_aio alone stays
genuinely no-GIL without any override. The release workflow, Makefile,
and make-wheels.sh now also build cp313t/cp314t wheels alongside the
existing versions (Windows ships pure-Python only, already
version-agnostic). None of the three C extensions declare Py_mod_gil
support yet, so a real free-threaded install still needs PYTHON_GIL=0
to keep the GIL off after importing one - a possible follow-up once
this is battle-tested.
CI's 3.13t/3.14t jobs now force PYTHON_GIL=0 so they actually exercise
no-GIL concurrency for every backend, not just build-and-import.

The release workflow, Makefile, and make-wheels.sh build cp313t/cp314t
wheels alongside the existing versions on every platform that builds C
extensions at all (Windows ships pure-Python only, already
version-agnostic). macOS switches from actions/setup-python to uv for
interpreter provisioning, since it's the one already confirmed to
handle free-threaded version strings correctly.
test_high_concurrency_stress_no_data_races failed on Windows CI at a
constant "chunk 10" - the only chunk whose entire payload is the raw
byte 0x0A ('\n'). os.open() defaults to text mode on Windows without
os.O_BINARY, so every '\n' silently became '\r\n' on write and back on
read - not a race at all, just a missing flag. Fixed in the test.

While investigating, found and fixed a real (if harder to trigger)
bug along the way: python_aio's Windows-only pread/pwrite fallback
locked per-fd via `self._locks[fd]` on a `defaultdict(RLock)` - two
threads racing on the same fd's first access could each create and
store their own RLock instance, ending up serialized against two
different objects instead of each other. Replaced with an explicit
get-or-create under the (previously declared but unused)
_locks_cleaner lock, and switched the dict to a WeakValueDictionary
so a long-lived Context doesn't accumulate one RLock per fd forever.

Verified directly on the Windows box this originally failed on:
reproduced pre-fix, clean across 5 repeated runs post-fix.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens caio’s four backends for genuinely free-threaded (no-GIL) CPython by removing GIL-dependent races in operation claiming, callback handling, and (for io_uring) ring bookkeeping, and adds CI/release support plus targeted stress tests to prevent regressions.

Changes:

  • Make Operation claiming and completion/result publication safe under true parallelism (atomics / critical sections in C backends; per-Operation lock in python_aio).
  • Add a comprehensive free-threading stress/regression test suite and expand CI to exercise PYTHON_GIL=0 on free-threaded interpreters.
  • Build and publish cp313t/cp314t wheels across build/release automation (Makefile, scripts, workflows, metadata classifiers).

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/test_free_threading.py Adds no-GIL regression + stress tests across backends and concurrency scenarios.
scripts/make-wheels.sh Builds additional free-threaded wheel tags (cp313t, cp314t).
pyproject.toml Declares the “Free Threading” classifier in project metadata.
Makefile Adds macOS venv/build targets for 3.13t/3.14t.
caio/thread_aio.c Uses atomics/critical sections to prevent cross-context claim races and callback/payload publication races; declares free-threaded support.
caio/python_aio.py Adds per-Operation locking to fix cross-context claim/callback races; adjusts fd-lock bookkeeping.
caio/linux_uring.c Adds critical sections around SQ/CQ ring bookkeeping; makes claim/callback/done publication thread-safe; declares free-threaded support.
caio/linux_aio.c Makes claim/callback/done publication thread-safe; declares free-threaded support.
.github/workflows/publish.yml Publishes cp313t/cp314t wheels and updates macOS wheel build to use uv.
.github/workflows/ci.yml Adds free-threaded test matrix coverage (with PYTHON_GIL=0) plus a python-only free-threaded validation job.
Suppressed comments (1)

tests/test_free_threading.py:95

  • The finally block unconditionally calls ctx.close(), but ctx may be None/unset if Context construction failed. Guard the close so cleanup can’t raise and hide the original exception.
    finally:
        os.close(fd)
        ctx.close()


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_free_threading.py
Comment thread caio/thread_aio.c
plot_results.py now loads two runs (bench_all.csv.gz / bench_all_nogil.csv.gz)
and stacks the GIL/no-GIL charts for each figure, so the actual throughput
and latency impact of running under a genuinely free-threaded interpreter
is visible directly alongside the normal GIL baseline. Charts moved under
benchmark/results/ alongside the raw data they're generated from.
- test_high_concurrency_stress_no_data_races: init ctx = None before
  the try so finally's ctx.close() can't itself raise UnboundLocalError
  and mask a genuine Context() construction failure.
- thread_aio.c: payload getter's comment claimed a completed write's
  py_buffer is freed right after its callback runs - worker() actually
  clears it before publishing done and before the callback, matching
  the surrounding "publish completion only after every write completes"
  comment. Code was already correct, only the comment was stale.
ConcurrentThreads (worker threads that surface failures on the main
thread) and a submit_and_wait fixture (submit N ops, wait for every
callback, collect results) factor out the repeated boilerplate that
had accumulated across tests/test_free_threading.py's stress tests.
@mosquito
mosquito merged commit 12f3408 into master Aug 4, 2026
22 checks passed
@mosquito
mosquito deleted the free-threading branch August 4, 2026 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for free-threaded (no-GIL) CPython

3 participants