Add robustness and sensitivity analysis

[DiogoRibeiro7]

Sweeps the analytical choices a QCA result is conditional on, and reports which paths survive.

analysis = robustness_analysis(
    data, outcome="SURV", conditions=[...],
    grid=RobustnessGrid(consistency=[0.75, 0.8, 0.85], pri=[0.5, 0.6], frequency=[1, 2]),
)
analysis.stable_terms()        # survive nearly every cutoff
analysis.fragile_terms()       # appear only under some
analysis.disappearing_terms()  # in the baseline, do not survive
analysis.emerging_terms()      # stable, but the baseline missed them

All four axes from the plan are covered: consistency, PRI and frequency cutoffs through robustness_analysis, and calibration anchors through calibration_robustness, which recalibrates from raw measures for each anchor combination.

Failures are recorded, not dropped

A specification that produces no solution gets a row with a failure message and NaN fit rather than vanishing. "The model collapses above 0.9" is a finding, and silently omitting those rows would make the survivors look more robust than they are.

Four similarity scales

Textual identity is the strictest comparison and often the least informative. solution_similarity also reports term overlap, configurational overlap over the literals used, and a fuzzy Jaccard over case membership. The last catches agreement the text hides: A and A+A*B are textually different and select exactly the same cases, scoring 1.0 on membership — there is a test for precisely that.

The framework refuses to overclaim

A stable path is stable, not true. Stability says the finding does not depend on one arbitrary cutoff; it says nothing about causal relevance, calibration sensibility, case selection, or omitted conditions. A mis-specified model can be perfectly stable, because sweeping cutoffs cannot detect a problem that lives in the model. The module docstring, the guide and the printed report all say so, and a test asserts the report keeps saying so.

Notes

Two report branches depend on the sweep happening to produce a disappearing or emerging term, so they are pinned against hand-built analyses rather than left to chance — as is the all-stable case where the report should stay quiet.

434 tests at 100% coverage; mypy strict and docs build clean.

---

Summary by cubic

Adds robustness and sensitivity analysis for QCA by sweeping consistency/PRI/frequency thresholds and calibration anchors, then reporting which solution terms persist across specifications. Also adds solution similarity measures, records failing specs, and includes a new guide.

• New Features
  • RobustnessGrid to sweep consistency, PRI, frequency; optional anchors for calibration sweeps.
  • robustness_analysis for threshold sweeps on calibrated data; returns RobustnessAnalysis.
  • calibration_robustness recalibrates from raw data per anchor combo (outcome_anchors, base_anchors).
  • Stability buckets: stable_terms, fragile_terms, disappearing_terms, emerging_terms (threshold adjustable).
  • solution_similarity with identity, term overlap, configurational, and membership comparisons.
  • Failing specs are kept with a failure message and NaN fit; to_frame() exports all runs.
  • Supports conservative, parsimonious, and intermediate (needs expectations) solution families.
  • Exposes RobustnessGrid, RobustnessAnalysis, robustness_analysis, calibration_robustness, and solution_similarity from package root; added guide guide/robustness.md.

^{Written for commit 65fd74b. Summary will update on new commits.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!