Skip to content

enh: warn loudly when no latent variable is significant and reliable - #22

Merged
eilidhmacnicol merged 5 commits into
mainfrom
enh/warn-no-significant-lvs
Jun 4, 2026
Merged

enh: warn loudly when no latent variable is significant and reliable#22
eilidhmacnicol merged 5 commits into
mainfrom
enh/warn-no-significant-lvs

Conversation

@eilidhmacnicol

Copy link
Copy Markdown
Contributor

Follow-up: make the null result unmissable

When a run finds no latent variable that is both significant (p < 0.05) and reliable (|bootstrap ratio| > 1.96 on both sides), run_pipeline completes normally — which is correct, a null result is a legitimate outcome. But it was communicated far too quietly:

  • only an empty significant_lvs: [] in log.txt and an INFO line,
  • a subject_scores.csv containing just the subject-ID index with no score columns,
  • the per-LV score, loading, and bootstrap-ratio plots silently absent.

A user skimming the figures/ folder just saw missing plots with no explanation. This adds an explicit WARNING at the end of the run when final_lv_names is empty, spelling out the criteria and that those plots were skipped. The graceful-completion behaviour is unchanged.

Changes

  • pipeline.run_pipeline: branch the end-of-run summary — INFO listing the LVs when any survive, a WARNING describing the null result when none do.

Scope note

The cross-validation pipeline does not have the same silent-null problem — it always reports accuracy, a permutation p-value, and figures regardless of significance, so there is no skipped output to warn about. Flagging "doesn't beat chance" there would stray into interpretation, so it's deliberately left alone.

Tests (written first, watched fail)

  • TestNullResultWarning::test_warns_when_no_lvs_survive — forces an all-false final_lvs (monkeypatching PLS.filter_lvs) and asserts a WARNING record is emitted.
  • test_no_warning_when_lvs_survive — a normal run (which keeps ≥1 LV on the synthetic data) emits no such warning.

All 183 tests pass; ruff clean; coverage 98%.

🤖 Generated with Claude Code

@eilidhmacnicol
eilidhmacnicol merged commit 844b9bd into main Jun 4, 2026
4 checks passed
@eilidhmacnicol
eilidhmacnicol deleted the enh/warn-no-significant-lvs branch June 4, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant