From 8e3b4e4ffb1b06f9546028224963a3ce6fa8e63f Mon Sep 17 00:00:00 2001 From: Nicholas Ceglia Date: Sun, 12 Jul 2026 09:15:18 -0400 Subject: [PATCH 1/2] refactor(pr2): delete 14 dead / out-of-scope symbols (Phase 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Safe deletions — every target verified to have zero in-package call-sites before removal (incl. precise call-site check for _ent / module-level dkl / probabilities). AST-span removal of top-level defs + SankeyNode.hex_to_rgb + the dead `probabilities` import. 384 lines removed; import tcri green; suite 35 passed / 1 skipped (nothing referenced them). Removed: pp.get_latent_embedding, pp.group_small_clones, pp.register_probability_columns, pp.remove_meaningless_genes, pp.gene_entropy, pp.classify_phenotypes, pl.polar_plot, pl.probability_distribution, pl.bayesian_mutual_information, metrics._ent, tl.clone_fraction, metrics.dkl, ut.probabilities, SankeyNode.hex_to_rgb. Co-Authored-By: Claude Opus 4.8 --- docs/contract/REFACTOR_AGENDA.md | 25 +-- tcri/metrics/_metrics.py | 20 --- tcri/plotting/_plotting.py | 225 --------------------------- tcri/plotting/_sankey.py | 3 - tcri/preprocessing/_preprocessing.py | 129 +-------------- tcri/utils/_utils.py | 9 -- 6 files changed, 18 insertions(+), 393 deletions(-) diff --git a/docs/contract/REFACTOR_AGENDA.md b/docs/contract/REFACTOR_AGENDA.md index 09825c5..9063977 100644 --- a/docs/contract/REFACTOR_AGENDA.md +++ b/docs/contract/REFACTOR_AGENDA.md @@ -33,7 +33,7 @@ tracker + running diary for the whole refactor. The detailed spec lives in `tcri |---|---|---|---|---|---| | 0 | Contract freeze + CI scaffolding | ✅ | none | — | conformance green | | 1 | Shared helpers + `_keys` | ✅ | low | 0 | existing tests green | -| 2 | Safe deletions | ☐ | very low | 1 | import-graph clean | +| 2 | Safe deletions | ✅ | very low | 1 | import-graph clean | | 3 | Model module split | ☐ | low | 1 | model/pyro tests green | | 4 | Model→AnnData streamline | ☐ | HIGH | 1,3 | session round-trip | | 5 | Engine consolidation | ☐ | HIGH | 4 | joint identities | @@ -47,12 +47,12 @@ tracker + running diary for the whole refactor. The detailed spec lives in `tcri ## Removal Ledger (the hard bar — every one MUST end deleted) Tick only when the symbol is gone from source AND `__all__`/imports AND `import tcri` is green. -**Phase 2 (dead / out-of-scope):** -- [ ] `pp.get_latent_embedding` · [ ] `pp.group_small_clones` · [ ] `pp.register_probability_columns` -- [ ] `pp.remove_meaningless_genes` · [ ] `pp.gene_entropy` · [ ] `pp.classify_phenotypes` -- [ ] `pl.polar_plot` · [ ] `pl.probability_distribution` · [ ] `pl.bayesian_mutual_information` -- [ ] `metrics._ent` · [ ] `tl.clone_fraction` · [ ] `metrics.dkl` (→ `_distance.kl_divergence`) -- [ ] `ut.probabilities` (+ its `_plotting.py:18` import, same PR) · [ ] `SankeyNode.hex_to_rgb` +**Phase 2 (dead / out-of-scope):** ✅ ALL DELETED (PR2) — 14 symbols, 384 lines, `import tcri` green. +- [x] `pp.get_latent_embedding` · [x] `pp.group_small_clones` · [x] `pp.register_probability_columns` +- [x] `pp.remove_meaningless_genes` · [x] `pp.gene_entropy` · [x] `pp.classify_phenotypes` +- [x] `pl.polar_plot` · [x] `pl.probability_distribution` · [x] `pl.bayesian_mutual_information` +- [x] `metrics._ent` · [x] `tl.clone_fraction` · [x] `metrics.dkl` (→ `_distance.kl_divergence`) +- [x] `ut.probabilities` (+ its `_plotting.py` import, same PR) · [x] `SankeyNode.hex_to_rgb` **Phase 4 (folded into `to_anndata` / session):** - [ ] `pp.register_model` (→ `model.to_anndata`) · [ ] `pp.register_phenotype_key` · [ ] `pp.register_clonotype_key` @@ -106,8 +106,15 @@ Template per PR: **Goal · Status · What happened · Issues & fixes · Added - **Usability:** internal only this step. - **`K.*` migration (done):** replaced **85** canonical key literals with `K.*` across preprocessing/metrics/plotting/utils via a verified script (model=0; only legacy keys there); added `test_no_canonical_key_literals` guard — none remain. `dkl` reassigned (dead `metrics.dkl`→Phase 2; `flux` inner→Phase 6). Legacy keys left as literals until their removal phases. **PR1 COMPLETE — 35 passed / 1 skipped.** -## PR 2 — Safe deletions · ☐ todo -_(diary to be filled — this is a REMOVAL PR; the ledger Phase-2 block must be fully ticked)_ +## PR 2 — Safe deletions · ✅ done (branch `refactor/pr2-safe-deletions`) +- **Goal:** delete the Phase-2 dead / out-of-scope symbols outright (they go to the trash, not to examples). Zero behavior change to the kept surface; `import tcri` stays green. +- **What happened:** verified **every** target has zero in-package call-sites (grep for calls/imports, plus a precise `NAME(` call-site check for the three ambiguous ones — `_ent`, module-level `dkl`, `probabilities` — all 0), then deleted via AST-span (top-level `FunctionDef` line ranges; the `SankeyNode.hex_to_rgb` method inside its `ClassDef`; the dead `from ..utils._utils import probabilities` line). **14 symbols, 384 lines removed** across metrics/plotting/sankey/preprocessing/utils. `import tcri` green; **full suite 35 passed / 1 skipped** (unchanged — nothing referenced them). +- **Why safe (not deferred):** the "dkl" refs that remain are the **string** distance-metric name in `flux` + the `_distance` registry, not the deleted module-level `dkl` function (`flux` keeps its own inner `dkl_func` → Phase 6). The `polar_plot`/`probability_distribution` "callers" were self-referential (own docstring examples / the self-recursion bug). `probabilities` was imported into plotting but never called. +- **Added:** n/a (pure removal PR). +- **Removed (hard bar):** ✅ all 14 Phase-2 ledger items ticked — `pp.get_latent_embedding`, `pp.group_small_clones`, `pp.register_probability_columns`, `pp.remove_meaningless_genes`, `pp.gene_entropy`, `pp.classify_phenotypes`, `pl.polar_plot`, `pl.probability_distribution`, `pl.bayesian_mutual_information`, `metrics._ent`, `tl.clone_fraction`, `metrics.dkl`, `ut.probabilities` (+ import), `SankeyNode.hex_to_rgb`. Confirmed gone: no `def` remains, no explicit re-export/`__all__` names them. +- **Test opportunities:** none new (removal); the existing suite is the regression gate and stayed green. +- **Streamline:** shrinks preprocessing (−129) and plotting (−225) meaningfully ahead of the Phase 3/4/7 splits. Left module-top imports untouched (conservative — genuinely-orphaned imports get swept when each file is split/finalized; the PR1 audit already handled the utils ones). +- **Usability:** removes broken/dead public entry points (`probability_distribution` self-recursion, `bayesian_mutual_information` bad kwarg, `polar_plot` undefined-name) from the surface so nobody trips on them. ## PR 3 — Model module split · ☐ todo ## PR 4 — Model→AnnData streamline · ☐ todo diff --git a/tcri/metrics/_metrics.py b/tcri/metrics/_metrics.py index 6ff3d51..2f28b26 100644 --- a/tcri/metrics/_metrics.py +++ b/tcri/metrics/_metrics.py @@ -147,11 +147,6 @@ def mi_compare(adata, groupby, groups=None, treatment=None, n_samples=50, } -def dkl(p, q): - epsilon = 1e-10 - p = np.clip(p, epsilon, 1) - q = np.clip(q, epsilon, 1) - return entropy(p, q) import numpy as np, pandas as pd from scipy.stats import entropy # Shannon entropy @@ -165,11 +160,6 @@ def dkl(p, q): GRN="\x1b[32m"; CYN="\x1b[36m"; MAG="\x1b[35m"; YLW="\x1b[33m" # small helper ---------------------------------------------- -def _ent(p, base=2): - p = np.asarray(p, dtype=float) - eps = 1e-15 - p = p.clip(eps) / p.sum() - return entropy(p, base=base) def clonotypic_entropy_base( adata, @@ -609,16 +599,6 @@ def clonality(adata): entropys[phenotype] = np.nan_to_num(clonality) return entropys -def clone_fraction(adata, groupby): - frequencies = dict() - for group in set(adata.obs[groupby]): - frequencies[group] = dict() - sdata = adata[adata.obs[groupby] == group] - total_cells = len(sdata.obs.index.tolist()) - clones = sdata.obs[sdata.uns["tcri_clone_key"]].tolist() - for c in set(clones): - frequencies[group][c] = clones.count(c) / total_cells - return frequencies def mutual_information( diff --git a/tcri/plotting/_plotting.py b/tcri/plotting/_plotting.py index 1a17053..cd0fce6 100644 --- a/tcri/plotting/_plotting.py +++ b/tcri/plotting/_plotting.py @@ -16,7 +16,6 @@ import operator import itertools -from ..utils._utils import probabilities from ._sankey import SankeyNode from ..preprocessing._preprocessing import clone_size, joint_distribution, joint_distribution_posterior from ..metrics._metrics import clonotypic_entropy as centropy @@ -514,26 +513,6 @@ def phenotypic_flux( if save is not None: fig.savefig(save) -def probability_distribution(adata, phenotype_order=None, color="#000000", rotation=90, splitby=None, order=None, figsize=(7,5), save=None): - columns = [] - if splitby != None: - ncols = len(set(adata.obs[splitby])) - else: - ncols = 1 - fig, ax = plt.subplots(1,ncols,figsize=figsize) - - if order == None: - order = list(sorted(adata.obs[splitby])) - for i, o in enumerate(order): - zdata = adata[adata.obs[splitby] == o] - pdist = probability_distribution(zdata) - sns.barplot(data=pdist,ax=ax[i],order=phenotype_order,color=color) - ax[i].set_xticklabels(ax[i].get_xticklabels(), rotation=rotation) - ax[i].set_title(o) - - fig.tight_layout() - if save != None: - fig.savefig(save) def top_clone_umap(adata, reduction="umap", top_n=10, fg_alpha=0.9, fg_size=25, bg_size=0.1, bg_alpha=0.6, figsize=(12,5), return_df=False,save=None): @@ -1220,210 +1199,6 @@ def mutual_information(adata, splitby=None, temperature=1.0, n_samples=0, normal return ax -def bayesian_mutual_information( - adata, - *, - group1, - group2, - splitby, - n_samples = 200, - temperature = 1.0, - normalised = True, - normalise_mode = "average", - weighted = False, - posterior = True, - combine_with_logits=True, - seed = 42, - palette = None, -): - np.random.seed(seed) - - meta = adata.uns[K.METADATA] - cov_col = meta["covariate_col"] - clone_col = meta["clone_col"] - - groups = sorted(adata.obs[splitby].dropna().unique().tolist()) - if palette == None: - palette = dict() - for i, g in enumerate(groups): - palette[g] = tcri_colors[i] - print(f"{BOLD}{MAG}──────── MI summary ({group1} → {group2}) ────────{RESET}") - _info("split column", splitby) - _info("# groups", len(groups)) - _info("Δ samples / group", n_samples) - _info("weighted", weighted) - print(f"{MAG}────────────────────────────────────────────────────{RESET}") - - results = {} - - # ---------- iterate over strata -------------------------------- - for g in groups: - mask_g = adata.obs[splitby] == g - clones = adata.obs.loc[mask_g, clone_col].unique().tolist() - - mi_pre = []; mi_post = [] - bar = tqdm(range(n_samples), desc=f"Δ-MI samples ({g})") - for _ in bar: - mi_pre.append( mutual_information_tl( - adata, group1, temperature=temperature, n_samples=1, - clones=clones, weighted=weighted, normalised=normalised, - normalise_mode=normalise_mode, posterior=posterior, - combine_with_logits=combine_with_logits, verbose=False)) - mi_post.append( mutual_information_tl( - adata, group2, temperature=temperature, n_samples=1, - clones=clones, weighted=weighted, normalised=normalised, - normalise_mode=normalise_mode, posterior=posterior, - combine_with_logits=combine_with_logits, verbose=False)) - - mi_pre = np.array(mi_pre) - mi_post = np.array(mi_post) - delta = mi_post - mi_pre - - d_mean, d_std = delta.mean(), delta.std() - hdi_low, hdi_hi = np.percentile(delta, [2.5,97.5]) - p_gt = (delta>0).mean(); p_lt = 1-p_gt - cohens_d = d_mean/d_std if d_std>0 else 0.0 - - print(f"\n{BOLD}{g}{RESET} ΔMI = {d_mean:.4f} ± {d_std:.4f} " - f"95 % HDI [{hdi_low:.4f}, {hdi_hi:.4f}] " - f"P(>0)={p_gt:.3f}") - - results[g] = dict(delta_samples=delta, mi_pre_samples=mi_pre, - mi_post_samples=mi_post, delta_mean=d_mean, - delta_std=d_std, cohens_d=cohens_d, - p_greater=p_gt, p_less=p_lt, - hdi=(hdi_low, hdi_hi)) - - fig, ax = plt.subplots(1, 3, figsize=(15, 4), - gridspec_kw=dict(width_ratios=[2, 2, 1]), - constrained_layout=True) - - # A ─── Δ-MI KDEs - for i, g in enumerate(groups): - sns.kdeplot(results[g]["delta_samples"], - fill=True, ax=ax[0], - palette=[palette[g]], - alpha=.9, linewidth=1.2, label=g) - ax[0].axvline(0, color="k", ls="--") - ax[0].set(title="Δ MI (post – pre)", - xlabel="Δ normalised MI", ylabel="density") - ax[0].legend(title=splitby) - - # B ─── pre / post KDEs - for i, g in enumerate(groups): - sns.kdeplot(results[g]["mi_pre_samples"], - ax=ax[1], palette=[palette[g]], - ls="-", label=f"{g} – {group1}") - sns.kdeplot(results[g]["mi_post_samples"], - ax=ax[1], palette=[palette[g]], - ls="--", label=f"{g} – {group2}") - ax[1].set(title="MI posterior per condition", - xlabel="normalised MI") - ax[1].legend() - - # C ─── bar summary of Δ - means = [results[g]["delta_mean"] for g in groups] - errs = [results[g]["delta_std"] for g in groups] - ax[2].bar(groups, means, yerr=errs, capsize=5, - color=[palette[g] for g in groups]) - ax[2].axhline(0, color="k", ls="--") - ax[2].set(title="Δ MI summary", ylabel="Δ normalised MI") - - fig.suptitle("Bayesian MI Analysis of clonotype ⇄ phenotype coupling", - fontsize=14, weight="bold") - return results -def polar_plot(adata, phenotypes=None, statistic="distribution", method="joint_distribution", splitby=None, color_dict=None, temperature=1.0): - """ - Create a polar plot showing phenotype distributions or entropies. - - This function creates a radar/polar chart that visualizes either the distribution of phenotypes - or entropy values across different conditions. It's useful for comparing phenotype proportions - or entropy patterns across experimental groups. - - Parameters - ---------- - adata : AnnData - AnnData object containing the data with TCR and phenotype information - phenotypes : list, optional - List of phenotype names to include in the plot. If None, uses all phenotypes - defined in adata.uns['tcri_metadata']["phenotype_col"] - statistic : str, default="distribution" - Type of statistic to plot, one of "distribution" or "entropy" - method : str, default="joint_distribution" - Method to compute phenotype distributions, either "joint_distribution" (model-based) - or "empirical" (raw cell counts) - splitby : str, optional - Column name to split the data by. If None, uses the covariate column stored - in adata.uns['tcri_metadata']["covariate_col"] - color_dict : dict, optional - Dictionary mapping split categories to colors - temperature : float, default=1.0 - Temperature parameter for softening/sharpening distributions - - Returns - ------- - matplotlib.axes.Axes - The polar plot axis - - Examples - -------- - >>> import tcri - >>> # Basic phenotype distribution polar plot - >>> ax = tcri.pl.polar_plot(adata, statistic="distribution") - >>> - >>> # Entropy polar plot with custom colors - >>> color_dict = {"Day0": "#FF5733", "Day7": "#33FF57", "Day14": "#3357FF"} - >>> ax = tcri.pl.polar_plot(adata, statistic="entropy", color_dict=color_dict) - """ - if phenotypes is None: - phenotypes = adata.uns[K.METADATA]["phenotype_col"] - - if splitby is None: - splitby = adata.uns[K.METADATA]["covariate_col"] - - # Get unique splits - splits = adata.obs[splitby].unique() - - # Create figure and axis - fig, ax = plt.subplots(figsize=(8, 8), subplot_kw={'projection': 'polar'}) - - # Calculate angles for each phenotype - angles = np.linspace(0, 2*np.pi, len(phenotypes), endpoint=False) - - # Plot for each split - for i, split in enumerate(splits): - if statistic == "distribution": - if method == "joint_distribution": - jd = joint_distribution(adata, split, temperature=temperature) - values = jd.mean().values - else: - subset = adata[adata.obs[splitby] == split] - values = np.zeros(len(phenotypes)) - for j, pheno in enumerate(phenotypes): - mask = subset.obs[adata.uns[K.METADATA]["phenotype_col"]] == pheno - values[j] = np.sum(mask) / len(subset) - else: # entropy - values = np.zeros(len(phenotypes)) - for j, pheno in enumerate(phenotypes): - values[j] = clonotypic_entropy(adata, split, pheno, temperature=temperature) - - # Normalize values - values = values / np.sum(values) - - # Plot values - color = color_dict[split] if color_dict else None - ax.plot(angles, values, 'o-', linewidth=2, label=split, color=color) - ax.fill(angles, values, alpha=0.25, color=color) - - # Set the labels - ax.set_xticks(angles) - ax.set_xticklabels(phenotypes) - - # Add legend - ax.legend(loc='upper right', bbox_to_anchor=(0.1, 0.1)) - - plt.tight_layout() - return ax \ No newline at end of file diff --git a/tcri/plotting/_sankey.py b/tcri/plotting/_sankey.py index 33f317f..6c6d03c 100644 --- a/tcri/plotting/_sankey.py +++ b/tcri/plotting/_sankey.py @@ -25,9 +25,6 @@ def plot(self, ax): ax.add_patch(self.patch) - def hex_to_rgb(self, hex_color): - hex_color = hex_color.lstrip('#') - return tuple(int(hex_color[i:i+2], 16) / 255.0 for i in (0, 2, 4)) def plot_node_connection(self, destination_node, ax, **kwargs): num_segments = 500 diff --git a/tcri/preprocessing/_preprocessing.py b/tcri/preprocessing/_preprocessing.py index 9cfc0eb..2a0aeea 100644 --- a/tcri/preprocessing/_preprocessing.py +++ b/tcri/preprocessing/_preprocessing.py @@ -83,47 +83,6 @@ def collapse_singleton(row): adata.obs[target_col] = adata.obs.apply(collapse_singleton, axis=1) -def classify_phenotypes(adata, phenotype_prob_slot="X_tcri_phenotypes", phenotype_assignment_obs=K.PHENOTYPE): - print("\t...classifying phenotypes...\n") - phenotype_col = adata.uns[K.METADATA]["phenotype_col"] - ct_array = adata.uns[K.CT_ARRAY] - unique_cts = np.unique(ct_array) - phenotype_probs_posterior = adata.uns[K.P_CT] - phenotypes = adata.uns[K.PHENOTYPE_CATEGORIES] - latent_z = adata.obsm[K.X_TCRI] - - # Pre-compute phenotype archetype embeddings - archetype_matrix = np.vstack([ - latent_z[adata.obs[phenotype_col].values == phenotype].mean(axis=0) - for phenotype in phenotypes - ]) - - all_probs = np.zeros((adata.n_obs, len(phenotypes))) - - # Iterate over each unique ct - for ct in unique_cts: - ct_indices = np.where(ct_array == ct)[0] - ct_embeddings = latent_z[ct_indices] - - # Cosine similarity (cells x phenotypes) - similarity = cosine_similarity(ct_embeddings, archetype_matrix) - similarity = (similarity + 1) / 2 # Normalize cosine similarity to [0,1] - - # Adjust similarity scores by posterior phenotype probabilities - adjusted_scores = similarity * phenotype_probs_posterior[ct] - - # Normalize to probabilities per cell - probs_normalized = adjusted_scores / adjusted_scores.sum(axis=1, keepdims=True) - all_probs[ct_indices] = probs_normalized - - # Store the normalized probabilities in AnnData - adata.obsm[phenotype_prob_slot] = all_probs - - # Assign phenotype with highest probability - assignments = all_probs.argmax(axis=1) - adata.obs[phenotype_assignment_obs] = pd.Categorical.from_codes( - assignments, categories=phenotypes - ) # ------------ helper to extract logits -------- # @torch.no_grad() @@ -319,38 +278,6 @@ def joint_distribution_posterior( _info("resulting DataFrame", df.shape, silent); _fin(silent) return df.round(precision) -def remove_meaningless_genes(adata, include_mt=True, include_rp=True, include_mtrn=True, include_hsp=True, include_tcr=True): - genes = [x for x in adata.var.index.tolist() if "RIK" not in x.upper()] - genes = [x for x in genes if "GM" not in x] - genes = [x for x in genes if "-" not in x or "HLA" in x] - genes = [x for x in genes if "." not in x or "HLA" in x] - genes = [x for x in genes if "LINC" not in x.upper()] - if include_mtrn: - genes = [x for x in adata.var.index.tolist() if "MTRN" not in x] - if include_hsp: - genes = [x for x in adata.var.index.tolist() if "HSP" not in x] - if include_mt: - genes = [x for x in genes if "MT-" not in x.upper()] - if include_rp: - genes = [x for x in genes if "RP" not in x.upper()] - if include_tcr: - genes = [x for x in genes if "TRAV" not in x] - genes = [x for x in genes if "TRAJ" not in x] - genes = [x for x in genes if "TRAD" not in x] - - genes = [x for x in genes if "TRBV" not in x] - genes = [x for x in genes if "TRBJ" not in x] - genes = [x for x in genes if "TRBD" not in x] - - genes = [x for x in genes if "TRGV" not in x] - genes = [x for x in genes if "TRGJ" not in x] - genes = [x for x in genes if "TRGD" not in x] - - genes = [x for x in genes if "TRDV" not in x] - genes = [x for x in genes if "TRDJ" not in x] - genes = [x for x in genes if "TRDD" not in x] - adata = adata[:,genes] - return adata.copy() def joint_distribution( adata, @@ -480,61 +407,9 @@ def joint_distribution( df_samples = df_samples[[col for col in df_samples.columns if col not in ["clonotype_id","clonotype_index","sample_id"]]] return df_samples -def get_latent_embedding( - adata, - latent_slot: str = K.X_TCRI, - n_samples: int = 0, - posterior_scale: float = 1.0 -) -> "np.ndarray": - mean_z = adata.obsm[latent_slot] - n_cells, latent_dim = mean_z.shape - samples = np.random.normal( - loc=mean_z, - scale=posterior_scale, - size=(n_samples, n_cells, latent_dim) - ) - return samples -def group_small_clones(adata, patient_key=""): - ct = [] - for x, s, p in zip(adata.obs["trb"], adata.obs[K.CLONE_SIZE], adata.obs[patient_key]): - if s < 4: - ct.append("Singleton_{}".format(p)) - else: - ct.append("{}_{}".format(x,p)) - adata.obs["trb_unique"] = ct - -def register_probability_columns(adata, probability_columns): - adata.uns["probability_columns"] = probability_columns - -def gene_entropy(adata, key_added="entropy", batch_key=None, agg_function=None): - import tqdm - if batch_key == None: - X = adata.X.todense() - X = np.array(X.T) - gene_to_row = list(zip(adata.var.index.tolist(), X)) - entropies = [] - for _, exp in tqdm.tqdm(gene_to_row): - counts = np.unique(exp, return_counts = True) - entropies.append(entropy(counts[1][1:])) - adata.var[key_added] = entropies - else: - if agg_function == None: - agg_function = np.mean - entropies = collections.defaultdict(list) - for x in tqdm.tqdm(list(set(adata.obs[batch_key]))): - sdata = adata[adata.obs[batch_key]==x] - X = sdata.X.todense() - X = np.array(X.T) - gene_to_row = list(zip(sdata.var.index.tolist(), X)) - for symbol, exp in gene_to_row: - counts = np.unique(exp, return_counts = True) - entropies[symbol].append(entropy(counts[1][1:])) - aggregated_entropies = [] - for g in adata.var.index.tolist(): - ent = agg_function(entropies[g]) - aggregated_entropies.append(ent) - adata.var[key_added] = aggregated_entropies + + def clone_size(adata, key_added=K.CLONE_SIZE, return_counts=False): tcr_key = adata.uns["tcri_clone_key"] diff --git a/tcri/utils/_utils.py b/tcri/utils/_utils.py index dd1c338..2a6096b 100644 --- a/tcri/utils/_utils.py +++ b/tcri/utils/_utils.py @@ -236,15 +236,6 @@ def load_tcri_session( -def probabilities(adata): - matrix = adata.obs[adata.uns["probability_columns"]] - barcodes = matrix.index.tolist() - cells = np.nan_to_num(matrix.to_numpy()) - index = adata.uns["joint_distribution"].index - probabs = dict() - for bc, cell in zip(barcodes, cells): - probabs[bc] = dict(zip(index, cell)) - return probabs tcri_colors = [ From 7fc2ecefa50563c78a0ac6b54361412fb863258b Mon Sep 17 00:00:00 2001 From: Nicholas Ceglia Date: Sun, 12 Jul 2026 09:21:11 -0400 Subject: [PATCH 2/2] =?UTF-8?q?refactor(pr2):=20audit=20fixes=20=E2=80=94?= =?UTF-8?q?=20drop=20orphaned=20import,=20correct=20doc=20counts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Multi-agent audit (3 lenses) verdict PASS with 3 LOW items, all fixed: - remove cosine_similarity import (orphaned by classify_phenotypes deletion) - diary preprocessing shrink -129 -> -127 (was double-counting blank residue) - plan: classify_phenotypes is Phase-2 DROP, not Phase-4 fold (per REDO_LIST) Co-Authored-By: Claude Opus 4.8 --- docs/contract/REFACTOR_AGENDA.md | 2 +- docs/contract/tcri_implementation_plan.md | 3 ++- tcri/preprocessing/_preprocessing.py | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/contract/REFACTOR_AGENDA.md b/docs/contract/REFACTOR_AGENDA.md index 9063977..a36eb93 100644 --- a/docs/contract/REFACTOR_AGENDA.md +++ b/docs/contract/REFACTOR_AGENDA.md @@ -113,7 +113,7 @@ Template per PR: **Goal · Status · What happened · Issues & fixes · Added - **Added:** n/a (pure removal PR). - **Removed (hard bar):** ✅ all 14 Phase-2 ledger items ticked — `pp.get_latent_embedding`, `pp.group_small_clones`, `pp.register_probability_columns`, `pp.remove_meaningless_genes`, `pp.gene_entropy`, `pp.classify_phenotypes`, `pl.polar_plot`, `pl.probability_distribution`, `pl.bayesian_mutual_information`, `metrics._ent`, `tl.clone_fraction`, `metrics.dkl`, `ut.probabilities` (+ import), `SankeyNode.hex_to_rgb`. Confirmed gone: no `def` remains, no explicit re-export/`__all__` names them. - **Test opportunities:** none new (removal); the existing suite is the regression gate and stayed green. -- **Streamline:** shrinks preprocessing (−129) and plotting (−225) meaningfully ahead of the Phase 3/4/7 splits. Left module-top imports untouched (conservative — genuinely-orphaned imports get swept when each file is split/finalized; the PR1 audit already handled the utils ones). +- **Streamline:** shrinks preprocessing (−127) and plotting (−225) meaningfully ahead of the Phase 3/4/7 splits. Removed the one import this PR orphaned (`cosine_similarity` — sole user was the deleted `classify_phenotypes`); other module-top imports left for the file-split phases (conservative; the PR1 audit already handled the utils ones). - **Usability:** removes broken/dead public entry points (`probability_distribution` self-recursion, `bayesian_mutual_information` bad kwarg, `polar_plot` undefined-name) from the surface so nobody trips on them. ## PR 3 — Model module split · ☐ todo diff --git a/docs/contract/tcri_implementation_plan.md b/docs/contract/tcri_implementation_plan.md index cfc7747..12cdc12 100644 --- a/docs/contract/tcri_implementation_plan.md +++ b/docs/contract/tcri_implementation_plan.md @@ -117,7 +117,8 @@ Freeze the map **before** Phase 5 (the first breaking PR). Renames are breaking; | `mi_compare`, `delta_entropy_table`, `flux_table`, `delta_clonotypic_entropy`, `phenotypic_entropy_delta` | **deleted** — expressed via `groupby=` + `tl.compare_groups` | Phase 6 | | `tl.phenotypic_entropies` / `tl.clonotypic_entropies` (plural batch forms) | **deleted** — subsumed by `groupby=` on the singular metric | Phase 6 | | `get_cell_phenotype_probs` | `predict` (scvi/CellAssign idiom; order‑preserving loader, indexed by `obs_names`) | Phase 4 | -| `register_model` (+ `classify_phenotypes`, `register_*_key`) | `model.to_anndata` (thin) | Phase 4 | +| `register_model` (+ `register_*_key`) | `model.to_anndata` (thin) | Phase 4 | +| `classify_phenotypes` | — (DROP; superseded by `REDO_LIST.md`) | Phase 2 ✅ deleted | | `register_clonotype_key` / `register_phenotype_key` | folded (private) into `to_anndata` | Phase 4 | | `pl.clonotypic_entropy_by_phenotype` | `pl.clonotypic_entropy` | Phase 7 | | `plot_pheno_sankey` | `pl.phenotypic_flux` (sankey) | Phase 7 | diff --git a/tcri/preprocessing/_preprocessing.py b/tcri/preprocessing/_preprocessing.py index 2a0aeea..a45fac0 100644 --- a/tcri/preprocessing/_preprocessing.py +++ b/tcri/preprocessing/_preprocessing.py @@ -17,7 +17,6 @@ import torch import torch.nn.functional as F from typing import Optional -from sklearn.metrics.pairwise import cosine_similarity import umap import numpy as np, pandas as pd, torch, umap from tqdm.auto import tqdm