Skip to content

chore: swap build backend from setuptools to maturin - #975

Open
SkyeAv wants to merge 2 commits into
mainfrom
feat/migrate-to-maturin-build-backend
Open

chore: swap build backend from setuptools to maturin#975
SkyeAv wants to merge 2 commits into
mainfrom
feat/migrate-to-maturin-build-backend

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Jul 27, 2026

Copy link
Copy Markdown
Member

Swaps the build backend from setuptools to maturin so Babel builds as a mixed Rust/Python
package, ahead of landing native (Rust) functionality. This is the backend swap only — the Rust
side is an inert placeholder module, the importable package stays src, and nothing under it
moves or renames.

Build backend

  • pyproject.toml: [build-system] is now requires = ["maturin>=1.0,<2.0"] /
    build-backend = "maturin"; dropped [tool.setuptools.packages.find].
  • [tool.maturin]: module-name = "src.rs" places the compiled extension at src/rs.* and
    tells maturin to bundle the src package (auto-resolved to the repo root — src-layout detection
    doesn't fire because there's no src/babel_pipeline/__init__.py); manifest-path = "rust/Cargo.toml" keeps Cargo off the Python src/ dir.
  • dev group: added maturin>=1.0,<2.0 so uv run maturin develop works locally (the
    [build-system].requires entry is build-isolation only).

Rust stub

  • rust/Cargo.toml + rust/src/lib.rs: empty #[pymodule] fn rs (pyo3, abi3-py311 +
    extension-module), importable as from src import rs. Real functionality lands in a follow-up.
  • Layout: Rust lives under rust/ so Cargo's default src/lib.rs never collides with the
    Python src/ package.

Build prerequisites (CI + Docker)

  • .github/workflows/test.yml: added dtolnay/rust-toolchain@stable before uv sync in both
    jobs — every uv sync now compiles via cargo, and ubuntu-latest ships no Rust.
  • Dockerfile: apt-get install -y cargo (system-wide, before USER nru) for the same reason.
  • Accepted caveat: a Rust toolchain is now a build prerequisite for everyone (local, CI,
    downstream installs) — the cost of adopting maturin before the Rust code exists.

Housekeeping

  • .gitignore: ignore rust/target/ and the maturin develop binaries
    (src/rs.*.so/.pyd/.dylib).
  • uv.lock + rust/Cargo.lock: regenerated and committed (CI runs uv sync --frozen).

Design

  • Why maturin needs a stub: maturin is fundamentally a Rust build tool — every layout it
    supports requires a Cargo.toml, so a bare backend swap with no Rust won't build. The empty
    #[pymodule] is the smallest thing that makes maturin viable now.
  • Why module-name = "src.rs": the import package is literally named src (flat layout).
    Verified against maturin's resolver (project_layout.rs / module_writer/mod.rs): this bundles
    all of src/* and drops the extension at src/rs.<so> without moving or renaming anything.
  • Deferred: real Rust functionality; a manylinux/abi3 publishing matrix (the stub builds a
    local cp311-abi3 wheel).

Testing

  • uv sync — builds babel-pipeline via maturin → cargo; installs maturin==1.14.1.
  • uv run python -c "import src; from src import rs; from src.tools.clique_diff import cli" — ok.
  • uv run pytest -m unit -q402 passed, 105 deselected.
  • uv buildbabel_pipeline-1.17-cp311-abi3-linux_x86_64.whl; unzip -l shows src/__init__.py,
    src/node.py, src/tools/..., src/rs.abi3.so, and all 4 console scripts in entry_points.txt.
  • uv run babel-clique-diff --help — resolves.

Questions for the reviewer

  • Rust toolchain as a prerequisite. Every build now needs cargo (local, CI, Docker). Fine for a
    stub-only PR, or would you rather gate the maturin switch behind the PR that adds real Rust?

Replace the setuptools build backend with maturin so the project builds as a
mixed Rust/Python package. This is the backend swap only — the Rust side is an
empty placeholder module (src.rs, importable as `from src import rs`) that real
functionality will fill in via a follow-up.

- pyproject.toml: build-system -> maturin; drop [tool.setuptools.packages.find];
  add [tool.maturin] (module-name = "src.rs", manifest-path = "rust/Cargo.toml");
  add maturin to the dev dependency group.
- rust/: new Cargo.toml + src/lib.rs stub (empty #[pymodule] `rs`, pyo3 abi3).
  Rust source lives under rust/ to avoid colliding with the Python src/ dir.
- .gitignore: ignore rust/target/ and the maturin-develop .so/.pyd/.dylib.
- CI/Docker: install a Rust toolchain (dtolnay/rust-toolchain in test.yml, apt
  cargo in Dockerfile) since every `uv sync` now compiles via cargo.
- uv.lock + rust/Cargo.lock regenerated.

The importable package stays `src`; nothing under src/ moves or renames, and all
console scripts and subpackages are preserved (verified: 402 unit tests pass;
wheel bundles src/* plus src/rs.abi3.so).
@SkyeAv SkyeAv added enhancement New feature or request Priority: High dependencies Pull requests that update a dependency file python:uv Pull requests that update python:uv code testing Related to the test suite or testing infrastructure github_actions Pull requests that update GitHub Actions code developer tooling Tooling & workflow that helps develop, debug, and validate Babel without changing pipeline outputs labels Jul 27, 2026
@SkyeAv

SkyeAv commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

This would allow me to expose rust code directly to python. This backend is the base commit I need before I can start refactoring in that direction

@SkyeAv SkyeAv self-assigned this Jul 28, 2026
@SkyeAv SkyeAv added the structural Changes to the structure of Babel in order to apply DRY and to make future development easier. label Jul 28, 2026
astral-sh/ruff-action@v3 floats to the latest ruff, now 0.16.0 (pyproject
pins only a floor, >=0.14.14). Ruff 0.16.0 formats Python code fences inside
*.md, which overlaps with the dedicated rumdl Markdown check and was failing
'Check Python formatting with ruff' on every open PR. Exclude *.md from ruff
format so ruff stays scoped to Python and Markdown formatting CI is
deterministic across ruff releases.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR switches Babel’s packaging/build backend from setuptools to maturin so the project can build as a mixed Python/Rust distribution, while keeping the importable Python package layout unchanged (src remains the top-level package). It adds a minimal Rust/PyO3 stub extension to satisfy maturin’s requirements and updates CI/Docker to include a Rust toolchain for builds.

Changes:

  • Swap PEP 517 build backend to maturin and configure it to bundle the existing src Python package while emitting an extension module at src/rs.*.
  • Add a placeholder Rust crate (rust/) with an empty PyO3 #[pymodule] importable as from src import rs.
  • Update CI and Docker build prerequisites to install Rust, and add ignores/locks for new build artifacts and dependencies.

Reviewed changes

Copilot reviewed 5 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pyproject.toml Switch build backend to maturin; configure [tool.maturin]; add dev dependency; adjust ruff formatter config.
uv.lock Record maturin in the dev dependency set and lock resolution.
rust/Cargo.toml Define the Rust crate and PyO3 dependency for the placeholder extension module.
rust/src/lib.rs Add the empty #[pymodule] fn rs stub importable from Python.
rust/Cargo.lock Commit the Rust dependency lockfile for reproducible cargo builds.
.github/workflows/test.yml Install a Rust toolchain in CI before uv sync so maturin/cargo builds can run.
Dockerfile Install cargo in the container image so uv sync can build the maturin extension.
.gitignore Ignore Rust build outputs and local maturin develop-produced extension binaries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Dockerfile
Comment on lines +40 to +42
# Rust toolchain — required because the build backend is maturin (compiles a native extension).
RUN apt-get update && apt-get install -y cargo

Comment on lines +20 to 21
- uses: dtolnay/rust-toolchain@stable
- run: uv sync --frozen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file developer tooling Tooling & workflow that helps develop, debug, and validate Babel without changing pipeline outputs enhancement New feature or request github_actions Pull requests that update GitHub Actions code Priority: High python:uv Pull requests that update python:uv code structural Changes to the structure of Babel in order to apply DRY and to make future development easier. testing Related to the test suite or testing infrastructure

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

2 participants