Consolidate the per-cluster heatmaps #155

GiottoVisuals has four cluster-level heatmaps, and three of them are one plot with different values:

plotHeatmap() is genuinely different, because it shows per-cell values. The other three all start from per-cluster means (calculateMetaTable()) and differ only in what is plotted:

• gene means
• metadata means
• the correlation of the gene means

They also differ in conventions:

• Ordering arguments: cluster_custom_order in plotHeatmap() / dotPlot(), custom_cluster_order in plotMetaData*, and cluster_custom_order in showClusterHeatmap() as of showClusterHeatmap(cluster_custom_order =); accept any expression name #154.
• Drawing engine: ggplot for three of them, ComplexHeatmap for showClusterHeatmap().
• Gradient arguments: each function has its own.

What dotPlot() already does well

• The value shown is an argument: dot_size / dot_color accept any summary function.
• Values come through spatValues(), so feats can be genes, metadata columns or enrichment scores. That alone covers the plotMetaDataHeatmap() / plotMetaDataCellsHeatmap() split.
• Feature sets and panels: feats can be a named list of sets (one panel each), and group_by / group_by_subset split by a second variable, a general version of plotMetaDataHeatmap()'s second_meta_col.
• Ordering: one argument, cluster_custom_order.
• Gradients: one set of arguments (dot_color_gradient, gradient_midpoint, gradient_style, gradient_limits).

Its limit: arbitrary functions need per-cell values. It pulls cells × features into a data.table and summarizes per cluster. That's fine for a few dozen genes, but it can't use the one-pass grouped featStats path (analyzeData(featStatsParam, groups =)), so it doesn't scale to all features or stream on disk.

Proposal

One per-cluster heatmap over a shared means pass:

• Named statistics for the common cases: "mean", "pct_detected", "mean_detected", "sd". These come from grouped featStats' accumulators (sum, sum of squares, count detected, sum over detected) in one streaming pass.
• A summary function as the escape hatch, as in dotPlot(). It goes through spatValues() and per-cell values, for small feature sets or custom statistics.
• Values from spatValues(), so genes, metadata and enrichment columns all work.
• Correlation as an optional step on the means matrix (e.g. what = c("values", "correlation")), which makes showClusterHeatmap() a special case.
• Shared conventions: cluster_custom_order, named feature sets, group_by, dotPlot()'s gradient arguments, and one drawing engine.

dotPlot() could then use the named statistics for its defaults (dot_size / dot_color defaults are exactly "pct_detected" and "mean") and become streamable, with the function form still available.

The existing functions would be soft-deprecated toward the consolidated one. plotHeatmap() stays, since it's the per-cell view.

Related

A plot of a cluster tree's own correlation matrix fits better on the tree than here, e.g. a plot() method on Giotto's giottoTree with what = c("tree", "heatmap"), drawn from its recorded cor_matrix. There the branches always match the colors, because the matrix is the one that built the tree. The heatmaps above always recompute from the object, so they shouldn't draw a supplied tree's branches (see #154).