feat: add CCC, Huber, and CLIP losses with per-pathway validation met…

[BenjaminIsaac0111]

1. Core Loss Function Upgrades

• New Loss Criteria (losses.py):
  • CCCLoss: Added Concordance Correlation Coefficient loss to reward both correct pattern correlation and proper scaling.
  • CompositeLoss: Updated to allow configurable combinations of MSE/Huber and PCC/CCC..
• Unit Tests (test_losses.py): Added comprehensive testing for all the new loss criteria under masked and unmasked scenarios.

2. Validation & Metrics Enhancements

• Validation Engine (engine.py):
  • Overhauled validation to compute, log, and return both PCC and CCC metrics.
  • Added per-pathway tracking to output a breakdown of the top-3 best and worst pathways by CCC performance.
  • Added tracking for prediction variance (to detect representation collapse) and comparison against baseline MAEs.
• Trainer Integration (trainer.py): Integrated the updated metrics logging with the main training loop.

3. Exploratory SVG (Spatially Variable Genes) Analysis

• SVG Script (analyze_svg.py): Added a script to identify, map, and analyse spatially variable genes in the tissue samples.
• SVG Documentation (SVG_HEST_EXPLORATORY_ANALYSIS.md): Documented the SVG pipeline findings and methodology.

4. Recipes & Data Pipelines

• Pathway Activity Generation (compute_pathway_activities.py): Updated pathway activity computations.
• Dataset Loader (dataset.py): Upgraded data-handling structures for the HEST-1k dataset, supported by test_dataset.py.

5. CLI Arguments, Presets, and Checkpoints

• Configurability (arguments.py & builder.py):
  • Added CLI parameters to switch between mse / huber and pcc / ccc, and set CLIP weights/temperatures.
  • Presets in run_preset.py and trainer entry points in train.py were updated to handle these configuration pipelines.
• Checkpoint Serialisation (checkpoint.py): Updated serializers to correctly save and reload states, including new training args.

6. Visualization & UI

• Plots and Dash Callback Updates (visualisation.py & callbacks.py): Aligned visualisations and dashboard logic with the new model outputs.