Skip to content

Python Packaging & Distribution #35

Description

@yjmaxpayne

Overview

Tracks turning the current experimental, build-tree-only pybind11 layer into
a properly packaged, installable Python distribution — ergonomic API
surface, NumPy/CuPy buffer-protocol interop, a reproducible wheel build
across CUDA-enabled manylinux images, and publication to a package index.

Core Objectives

1. Python API ergonomics

snake_case naming, type stubs, docstrings, NumPy buffer-protocol-backed
array views for result data.

2. CuPy interoperability

Detect __cuda_array_interface__ to avoid unnecessary device-host round
trips for GPU-resident callers.

3. CI wheel build matrix

Across supported CPython versions × manylinux baseline × supported CUDA
major version.

4. Staged package index rollout

Test index first; public index once the build is verified stable across
independent environments.

5. Bare-container install smoke test

Install-from-wheel + import + minimal run, in a clean CUDA-enabled
container image.

Success Criteria

Metric Target
Wheel installable via pip in a clean container pass
NumPy array view over result data does not trigger an extra copy verified via memory-sharing check
CI wheel matrix green across all targeted CPython versions

Dependencies

Depends on Public API Evolution (a stable-enough surface to package) and
Backend Abstraction Architecture (a stable CUDA backend to ship). Hard
prerequisite for External Consumer Integration Contract's acceptance —
the consumer calls HELIX via the installed wheel, not a build tree.

Risks

manylinux + CUDA + pybind11 ABI combinations remain an immature packaging
path industry-wide. If CI debugging stalls, the fallback is to ship a
test-index-only build for a single platform target first and defer the full
matrix.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions