Optimized the Decontx-Python package - #1
Open
jjia1 wants to merge 7 commits into
Open
Conversation
GSD-Unit: Q1
…n_genes) Rewrites the four core numba functions in fast_ops.py to operate on CSR indptr/indices/data arrays, never allocating dense n_cells×n_genes matrices. model.py drops X.toarray() and passes CSR arrays directly; the final decontaminated counts are returned as a csr_matrix sharing the input sparsity pattern. core.py keeps layers sparse, adds flush=True to all prints for SLURM log visibility, and handles multi-batch sparse stacking. Tests updated and extended with sparse-specific assertions (sparsity pattern preservation, nonnegativity, per-cell totals, contamination range). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
anndata's h5ad writer has no serializer for Python tuples; _store_metadata was storing delta as-is (a Tuple[float,float] from the decontx() signature), causing IORegistryError on write_h5ad. Coercing to list at the storage site fixes serialization without changing the public API or EM behavior. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix batch_matrices key from tuple(batch_indices.tolist()) to batch_name - Remove dead row_parts computation (4 lines) - Store fitted delta (post-estimation) instead of input delta in uns metadata - Remove unused import pytest from test_core.py - Remove unused local import alias _csr Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gsd/small feature/small feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Core algorithmic rewrite: dense → sparse EM (6315838)
The most significant change rewrites the four core numba-accelerated EM functions in
fast_ops.py from dense matrix operations to sparse CSR arithmetic:
matrices on every iteration regardless of how sparse the data was. Memory use
scaled with the full matrix shape.
touching only nonzero entries. Complexity drops from O(n_cells × n_genes) to O(nnz)
per iteration — typically 10–100× fewer operations for sparse single-cell data.
The decontaminated output is also returned as a csr_matrix, keeping the sparsity
pattern intact.
model.py was updated to pass CSR arrays directly instead of densifying, and core.py
now handles multi-batch sparse stacking and keeps layers sparse throughout.