Skip to content

Add unique coverage and case-level sufficiency diagnostics - #14

Merged
DiogoRibeiro7 merged 5 commits into
mainfrom
feat/sufficiency-diagnostics
Aug 11, 2026
Merged

Add unique coverage and case-level sufficiency diagnostics#14
DiogoRibeiro7 merged 5 commits into
mainfrom
feat/sufficiency-diagnostics

Conversation

@DiogoRibeiro7

@DiogoRibeiro7 DiogoRibeiro7 commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Closes the unique-coverage gap the mathematical audit recorded, and adds the case typology that turns a solution into something you can select cases from.

diagnostics = sufficiency_diagnostics(data, outcome="SURV", terms=["DEV*URB", "LIT*~IND"])
diagnostics.terms[0].typical            # cases supporting the claim
diagnostics.terms[0].contradictory      # term holds, outcome does not
diagnostics.terms[0].deviant_coverage   # outcomes this term misses
diagnostics.terms[0].uniquely_covered   # cases no other term reaches
diagnostics.redundant_terms             # terms adding no unique coverage
diagnostics.to_frame(); diagnostics.cases_frame()

Terms are parsed from expression strings, so a solution can be pasted straight in. Case labels come from the index or a named column — no schema is assumed.

Unique coverage

covU_i = [ Σ min(Xᵢ,Y) − Σ min(Xᵢ, max_{j≠i} Xⱼ, Y) ] / Σ Y

Matches R's covU to 1e-9 for every multi-term solution on the Lipset data, along with per-term consistency, PRI and raw coverage. Two identical terms each get exactly zero, which is the degenerate case worth having a test for.

Two divergences from R, both characterised

Single-term solutions. R reports covU as NA — there is no other term to be unique against. setqca reports the raw coverage, since with nothing to share with, everything the term covers is uniquely covered by it. The parity test asserts the family really has one term before accepting this.

R's cases column is not "typical cases". It lists membership in the term above the crossover. The typology splits that set further: on the Lipset conservative solution R lists BE, CZ, NL, UK, and setqca agrees all four are in the term while additionally reporting UK as deviant-in-degree, its term membership exceeding its outcome membership. A parity test pins the correspondence as typical plus deviant-in-degree, not typical alone.

Case typology

Following Schneider and Rohlfing (2013), with a test dataset where all five roles are known by hand before the code runs. Only consistency deviance counts against the claim — a deviant-coverage case says the outcome arrived by another path, which is what a disjunctive solution expects, and CaseRole.contradicts_sufficiency encodes that.

391 tests at 100% coverage; mypy strict and docs build clean.


Summary by cubic

Adds per-term unique coverage (covU) and case-level sufficiency diagnostics for sufficiency solutions. Helps select cases, find contradictions, and identify redundant terms; matches R on Lipset data.

  • New Features
    • New sufficiency_diagnostics(data, outcome, terms, case_id=None) returns SolutionDiagnostics with per-term consistency, PRI, raw_coverage, unique_coverage, frequency, and per-case roles; includes .to_frame() and .cases_frame().
    • Case typology per Schneider & Rohlfing: typical, deviant consistency (kind/degree), deviant coverage, individually irrelevant; exposes CaseRole, CaseDiagnostic, and contradicts_sufficiency; accessors: typical, contradictory, deviant_consistency, deviant_coverage, uniquely_covered; flags redundant_terms.
    • Unique coverage implemented and validated: matches R within 1e-9 for multi-term solutions; for single-term solutions unique_coverage equals raw coverage.
    • Public exports from setqca: sufficiency_diagnostics, SolutionDiagnostics, TermDiagnostics, CaseRole, CaseDiagnostic; new guide page added and nav updated.
    • Validation: R fixtures now include per-term fits; parity tests cover consistency, PRI, raw_coverage, unique_coverage, and case membership; unit tests cover typology and overlap edge cases.

Written for commit 23319ed. Summary will update on new commits.

Review in cubic

@DiogoRibeiro7
DiogoRibeiro7 merged commit d622ac1 into main Aug 11, 2026
15 checks passed
@DiogoRibeiro7
DiogoRibeiro7 deleted the feat/sufficiency-diagnostics branch August 11, 2026 08:10
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant