retrieve_optimal_markers added in utils

[ivanferrari95]

🚀 Title

Add retrieve_optimal_markers: automated marker selection using CIA scoring

🧠 Summary

This merge request introduces a new function, retrieve_optimal_markers, to automatically identify optimal marker gene signatures for each group in a single-cell dataset.

The method combines differential expression analysis with CIA-like scoring and ROC AUC evaluation to select compact, high-performing gene signatures.

🎯 Motivation

Current workflows often rely on:

• manual inspection of differentially expressed genes
• arbitrary cutoffs such as selecting the top N genes
• a lack of quantitative evaluation of marker quality

This function provides a more data-driven and quantitative approach to:

• evaluate marker performance
• optimise signature size
• improve robustness across groups

⚙️ What the function does

For each group, the function performs the following steps:

1. Retrieve differentially expressed genes from scanpy.tl.rank_genes_groups
2. Filter genes based on:
   • log fold change
   • expression in the target group (pct1)
   • expression outside the target group (pct2)
   • mean expression
   • adjusted p-value
3. Rank genes by differential expression score and log fold change
4. Evaluate prefix gene signatures using:
   • CIA-like scoring
   • one-vs-rest ROC AUC
5. Select the optimal signature size:
   • either the signature with the best AUC
   • or the smallest signature within auc_tolerance of the best AUC
6. Return the selected marker genes for each group

🧬 Key Features

Quantitative marker evaluation

The function uses ROC AUC to quantify how well a gene signature separates one group from all others.

CIA-like scoring

The score combines:

• the number of detected genes
• the expression levels of those genes
• normalisation by total cellular expression

Optimal signature selection

The function avoids arbitrary selection of the top N genes and instead identifies a compact signature with strong discriminative performance.

Efficient implementation

To improve scalability, the function includes:

• parallel processing across groups via n_jobs
• search space reduction through:
  • a cap on the number of evaluated genes (max_genes)
  • coarse-to-fine prefix evaluation (step, refine_window)

🔧 Main Parameters

Parameter	Description
groupby	Column in adata.obs defining the groups
uns_key	Key containing differential expression results (default: rank_genes_groups)
logFC	Minimum log fold change threshold
pct1	Minimum percentage of expression in the target group
pct2	Maximum percentage of expression outside the target group
mean	Minimum mean expression threshold
padj	Adjusted p-value threshold
auc_tolerance	Allows smaller signatures with performance close to the best AUC
n_jobs	Number of parallel workers
max_genes	Maximum number of genes evaluated per group
step	Step size for coarse prefix scan
refine_window	Refinement window around the best prefix

📦 Output

The function returns a dictionary of marker genes in the form:

{group: [marker_genes]}