Add unique coverage and case-level sufficiency diagnostics - #14
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the unique-coverage gap the mathematical audit recorded, and adds the case typology that turns a solution into something you can select cases from.
Terms are parsed from expression strings, so a solution can be pasted straight in. Case labels come from the index or a named column — no schema is assumed.
Unique coverage
Matches R's
covUto 1e-9 for every multi-term solution on the Lipset data, along with per-term consistency, PRI and raw coverage. Two identical terms each get exactly zero, which is the degenerate case worth having a test for.Two divergences from R, both characterised
Single-term solutions. R reports
covUasNA— there is no other term to be unique against. setqca reports the raw coverage, since with nothing to share with, everything the term covers is uniquely covered by it. The parity test asserts the family really has one term before accepting this.R's
casescolumn is not "typical cases". It lists membership in the term above the crossover. The typology splits that set further: on the Lipset conservative solution R listsBE, CZ, NL, UK, and setqca agrees all four are in the term while additionally reporting UK as deviant-in-degree, its term membership exceeding its outcome membership. A parity test pins the correspondence as typical plus deviant-in-degree, not typical alone.Case typology
Following Schneider and Rohlfing (2013), with a test dataset where all five roles are known by hand before the code runs. Only consistency deviance counts against the claim — a deviant-coverage case says the outcome arrived by another path, which is what a disjunctive solution expects, and
CaseRole.contradicts_sufficiencyencodes that.391 tests at 100% coverage; mypy strict and docs build clean.
Summary by cubic
Adds per-term unique coverage (covU) and case-level sufficiency diagnostics for sufficiency solutions. Helps select cases, find contradictions, and identify redundant terms; matches R on Lipset data.
sufficiency_diagnostics(data, outcome, terms, case_id=None)returnsSolutionDiagnosticswith per-termconsistency,PRI,raw_coverage,unique_coverage,frequency, and per-case roles; includes.to_frame()and.cases_frame().typical,deviant consistency (kind/degree),deviant coverage,individually irrelevant; exposesCaseRole,CaseDiagnostic, andcontradicts_sufficiency; accessors:typical,contradictory,deviant_consistency,deviant_coverage,uniquely_covered; flagsredundant_terms.unique_coverageequals raw coverage.setqca:sufficiency_diagnostics,SolutionDiagnostics,TermDiagnostics,CaseRole,CaseDiagnostic; new guide page added and nav updated.consistency,PRI,raw_coverage,unique_coverage, and case membership; unit tests cover typology and overlap edge cases.Written for commit 23319ed. Summary will update on new commits.