Fix confirmed bugs from the issue tracker (#17, #18, #19, #26, #27)

[davisidarta]

Fixes five bugs reported in the tracker, each verified reproduced before and fixed after.
One commit per issue.

⚠️ This changes numerical output for cosine kernels. See #26 below — embeddings
computed with 1.1.0 using the default base_metric='cosine' will differ after this
merge. That is the fix working, not a regression, but it warrants a version bump and a
changelog note.

#26 — adaptive bandwidth in the wrong distance units (highest impact)

adap_sd was derived from the kNN graph's stored cosine distances (d = 1 - cos,
range [0, 2]), while the distances it normalises were converted to angles
([0, π]) only afterwards. dists / adap_sd[x] therefore divided radians by a
cosine-distance quantity.

This hit the default configuration — base_kernel_version='bw_adaptive',
base_metric='cosine', and use_angular force-enabled for cosine. Measured on 300×20
Gaussian data, k=10:

	master	fixed
normalised distance (mean)	2.3046	1.0690
mean kernel weight	0.0064	0.2840

A 2.16× inflation in the exponent of exp(-d²) collapses the weights. Neighbour
ordering survived because arccos is monotonic in d, which is why results looked
plausible instead of obviously wrong — and why this went unnoticed since 0168865
(2025-09-09), i.e. for all of 1.1.0.

The graph is now converted to angles before the bandwidth is derived, so one distance
convention holds throughout. This also fixes a second defect in the same block: in the
expand_nbr_search branch, dists_new got neither the conversion nor the [0, π] clip,
so use_angular was silently ignored there.

Verified: euclidean kernels are bit-identical to master; cosine kernels change as above.

#27 — nmslib dense index paired with a sparse space

NMSlibTransformer.fit() assigned the sparse space table unconditionally before checking
self.dense, so dense=True raised for every input, dense or sparse:

ValueError: The space type cosinesimil_sparse_fast is not compatible with
            the type DENSE_VECTOR, only dense vector spaces are allowed!

The dense space table was only reachable when dense=False, making the dense path dead
code. Space name and data type are now chosen together at one decision point.

kNN() also force-converted dense arrays to CSR under a warning claiming "nmslib does not
support dense matrices" — false, nmslib has DataType.DENSE_VECTOR — which additionally
said "Converting to array" while converting to sparse. Dense input now passes through.

Verified all four combinations of dense × sparse/dense input build with a correctly
paired space (cosinesimil vs cosinesimil_sparse_fast).

#19 — t-SNE hardcoded metric='precomputed'

The root cause was deeper than reported. TopOGraph.project() hands coordinate-based
methods the eigenbasis coordinates with metric=graph_metric, but the t-SNE branch
hardcoded metric='precomputed' — so t-SNE read coordinates as a square distance matrix.
It now honours self.metric.

That hardcoded metric is what surfaced the reported error: with precomputed, sklearn
≥ 1.2 rejects its own new default init='pca'. init='random' would have masked the real
bug, so the spectral initialisation TopOMetry already computes is passed instead.

Also: n_iter → max_iter (renamed in sklearn 1.5, removed in 1.7), selected by
introspection. On sklearn 1.9 the old call failed with TypeError before even reaching the
init check.

The MulticoreTSNE path is updated for consistency but untested — it does not build on
Python 3.12.

#18 — todense() returns np.matrix

eval_models_layouts() failed for sparse input because current sklearn rejects
np.matrix outright. Fixed at both sites (pipes.py, topograph.py) exactly as the
reporter suggested. utils/_utils.py already wraps its todense() in np.asarray.

#17 — trustworthiness not exported

Removed from local_scores.py in 4974acf in favour of sklearn's, but never re-exported
despite having been documented. Now available from topo.eval again.

Tests

New tests/test_kernel_units_and_backends.py (14 tests). Two fail on master for
behavioural rather than structural reasons — the weight-collapse assertion sees 0.0064
against a 0.05 threshold, and the dense nmslib index raises. nmslib cases skip
individually when the backend is absent, so kernel coverage survives without it.

Full suite: 39 passed (25 existing + 14 new), no regressions. Existing tests need
scikit-misc and hnswlib installed.

Not fixed here — reported separately

1. expand_nbr_search is a no-op. new_k = int(k + (k - pm.max())), and pm.max()
   is always exactly k by construction of the np.interp range, so new_k == k always.
   The feature recomputes the same graph and expands nothing. Choosing what new_k
   should be is a design decision, so it is left alone.
2. use_angular=False is silently ignored. The override at kernels.py:173 tests
   'use_angular' not in kwargs, but use_angular is a named parameter and so never
   appears in kwargs — passing it explicitly has no effect for cosine. Harmless given
   the corrected units are now the right default, but the flag does not work.
3. Test dependencies undeclared — the suite needs scikit-misc and hnswlib;
   scikit-misc is required by scanpy's seurat_v3 HVG flavor, which is TopOMetry's
   default.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4002338437

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Use the actual cosine similarity when angularizing

For the default compute_kernel(..., metric='cosine') path, K comes from kNN, which has already inverted sklearn/nmslib cosine distances with knn.data = 1 - knn.data, so K.data is cosine similarity rather than a cosine distance. Passing those values to _cosine_distance_to_angle computes arccos(1 - similarity): duplicate/self edges with similarity 1 become π/2 and opposite edges become 0, so the newly converted adaptive bandwidths and weights are still in the wrong units for cosine kernels. Convert from the stored similarity (arccos(K.data)) or stop inverting before this helper.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Densify sparse queries for dense nmslib indexes

When NMSlibTransformer(dense=True) is fit on a CSR matrix, this densifies only the local data used to build a DENSE_VECTOR index. The normal transformer flow (fit_transform(csr) or fit(csr).transform(csr)) still passes the original sparse matrix into transform, even though the nearby comment notes dense indexes cannot consume sparse rows, so the newly supported dense+sparse combination fails at query time. Record that the fit data was densified and apply the same conversion in transform and ind_dist_grad.

Useful? React with 👍 / 👎.