tet-py

Python bindings for Tetration — mmap-friendly .tet tensor files and the JSON/TOML query engine.

| Install (PyPI) | pip install tet-py | | Import | import tet | | Rust core | tetration on crates.io | | CLI (no Python) | tet binary from the main repo |

Status (0.1.1): read/query API (including preview=N on reductions); NumPy ram / spill / sidecar; TetWriter / write_dataset (f32/f64). Next: tet.convert extras, read_numpy preflight, integer write dtypes — see CHANGELOG.md and docs/HANDOFF.md.

Do not pip install tetration — that PyPI name is an unrelated math package. Use tet-py / import tet.

Quick start

Prerequisites

• Python 3.11+
• Rust 1.95+ (.mise.toml pins rust = "1.95", or install matching rustup toolchain; native/Cargo.toml sets rust-version = "1.95")
• maturin (pip install maturin or uv tool install maturin)

Development

cd ~/Code/tet-py
uv sync --extra dev
mise run develop   # or: uv run maturin develop
uv run python -c "import tet; print(tet.__version__, tet.core_version())"
uv run pytest -q
uv run mypy python/tet

Links tetration 0.1.9 from crates.io. Test fixtures live in tests/fixtures/.

Example

import tet

with tet.open("tests/fixtures/large.tet") as f:  # or any .tet path
    print(f.mean("a"), f.quantile("a", 0.5))
    r = f.mean("a", preview=32)                   # QueryResult: r.scalar + r.preview (ndarray)
    arr = f.read_numpy("a")                         # ram
    z = f.transform.to_numpy.zscore("a")            # transform → ram
    spill = f.transform.to_spill.zscore("a", path="a_zscore.bin")
    same = spill.to_numpy()                         # transform → spill → ndarray
    side = f.transform.to_sidecar.zscore("a", path="a_zscore.tet")
    same = side.to_numpy(f)                         # transform → sidecar .tet → ndarray

Operations reference (every op with examples): docs/operations.md

Topic	Where
mean, min, quantile, histogram, …	docs/operations.md
NumPy ram / spill / sidecar	docs/operations.md#read_numpy
build_query, selection_slices	docs/operations.md#selection-and-build_query
Wire schema / CLI	tetration query engine
All docs	docs/README.md

info() / summary()

Both return the full tet info --json dict (superblock, datasets, all chunk rows, metadata). For exploration, prefer list(f), f.dataset(name), or info["datasets"] — not printing the whole dict in the REPL.

Project layout

tet-py/
  pyproject.toml      # PEP 621 + maturin
  python/tet/         # public API + _core, _query, _io, _transform
  native/             # PyO3 extension (links tetration 0.1.9)
  tests/fixtures/     # vendored .tet fixtures for CI
  docs/               # operations.md, HANDOFF.md
  CHANGELOG.md

Roadmap

• Scaffold: maturin, tet.open, datasets, query (JSON document)
• query() / execute(raw=False) → QueryResult
• summary() / info() — dict parity with tet info --json
• Reduction helpers (mean, sum, min, max, …)
• quantile, histogram, covariance, correlation
• build_query, axis_slice, selection_slices
• Dataset, iter_datasets(), f[0] / f["name"], axis index or dim_names
• UnknownDatasetError / UnknownAxisError; optional typing_stub()
• mypy + _native.pyi
• NumPy read — ram (read_numpy, Dataset.to_numpy, transform.to_numpy)
• NumPy read — spill (read_spill, transform.to_spill, .to_numpy() loaders)
• NumPy read — sidecar (transform.to_sidecar, SidecarTransformResult.to_numpy)
• NumPy write (TetWriter, write_dataset)
• PyPI wheels (tet-py 0.1.1); tetration = "0.1.9" from crates.io
• Query preview samples (preview=N, QueryResult.preview) — #7
• CI fixtures vendored in tests/fixtures/ (no sibling tetration checkout) — #6

To do

Near-term

• read_numpy preflight (#9) — fail fast on oversized materialize (blocked on tetration #19)
• Integer write_dataset dtypes (#8) — beyond f32/f64
• tet.convert orchestration (#10) — optional extras (h5py, netCDF4, zarr, pandas, pyarrow); Rust tet convert stays the fast path when native libs are installed
• Zero-copy mmap → NumPy (#11) — views over mmap’d raw payloads (P2; after copy path stabilizes)

Docs

• Keep tetration-docs/python in sync with releases

Related

• CHANGELOG.md — release notes
• tetration-docs — Python — install, NumPy sinks, operations
• docs/operations.md — query ops (mean, quantile, …) with examples
• docs/README.md — doc index
• HANDOFF.md — phases, dev commands, agent notes
• Layout v1
• Query engine
• AGENTS.md — ops and phase status

License

Dual-licensed under MIT OR Apache-2.0, same as tetration.