Skip to content

Influence diagnostics: dffits + influence masks (#27 batch 3) - #30

Merged
sipemu merged 1 commit into
mainfrom
feature/issue-27-influence
May 28, 2026
Merged

Influence diagnostics: dffits + influence masks (#27 batch 3)#30
sipemu merged 1 commit into
mainfrom
feature/issue-27-influence

Conversation

@sipemu

@sipemu sipemu commented May 28, 2026

Copy link
Copy Markdown
Contributor

Batch 3 of 4 for #27. Adds 4 influence diagnostics as Polars expressions.

Implemented

Diagnostic Expression Default threshold Output shape
DFFITS `ps.dffits("y", "x1", ...)` struct{dffits: List, n_observations}
Influential Cook's `ps.influential_cooks("y", "x1", ...)` 4/n struct{is_influential: List, n_influential, n_observations}
Influential DFFITS `ps.influential_dffits("y", "x1", ...)` 2 * √(p/n) same
High leverage points `ps.high_leverage_points("x1", "x2", ...)` 2 * p/n same

All four follow the rlib split pattern (public *_fit + pl_* shim). Each fits OLS internally via the shared fit_ols_for_residual_diag helper introduced in batch 1, except high_leverage_points which only builds the design matrix (no y needed).

Tests

  • Pythontests/test_influence_diagnostics.py, 7 new pytests:
    • DFFITS at planted high-influence indices > 1.0.
    • influential_cooks and influential_dffits flag the planted indices.
    • Custom threshold tightens the influence mask.
    • high_leverage_points catches extreme x values without taking y.
  • Rusttests/rust_api.rs: smoke tests in diagnostic_fits for all 4 *_fit entry points.

Full Python suite: 448/448 pass (441 prior + 7 new). Rust integration: 12/12 pass.

Remaining work on #27

  • Batch 4 — multicollinearity + dispersion: generalized_vif, high_vif_predictors, classify_condition_number, variance_decomposition_proportions, pearson_chi_squared, estimate_dispersion_pearson, estimate_dispersion_deviance.
  • Probit/cloglog/negative_binomial/Tweedie residual variants (deferred from batch 2).
  • Standardized Pearson / standardized deviance residuals.
  • compute_leverage_with_aliased (variant for rank-deficient designs).

Test plan

  • cargo build --no-default-features clean
  • cargo clippy --all-targets -- -D warnings clean
  • cargo fmt --check clean
  • cargo test --no-default-features — 12/12 pass
  • pytest — 448/448 pass
  • CI matrix

🤖 Generated with Claude Code

Add 4 influence diagnostics following the established pattern:

  - dffits_fit + pl_dffits: scaled per-row influence on fitted value.
  - influential_cooks_fit + pl_influential_cooks: boolean mask of
    observations above Cook's distance threshold (default 4/n).
  - influential_dffits_fit + pl_influential_dffits: boolean mask of
    observations above DFFITS threshold (default 2 * sqrt(p/n)).
  - high_leverage_points_fit + pl_high_leverage_points: boolean mask
    of observations with leverage > threshold (default 2*p/n). Takes
    only X (no y).

Each fits OLS internally via fit_ols_for_residual_diag (except
high_leverage_points which only builds the design matrix).

Two new struct output schemas:
  dffits_output_dtype{dffits: List<Float64>, n_observations}
  influence_mask_output_dtype{is_influential: List<Boolean>,
                              n_influential, n_observations}

Python (exprs/regression.py): dffits, influential_cooks,
influential_dffits, high_leverage_points — all exported.

Tests:
  tests/test_influence_diagnostics.py: 7 pytests covering shape and
  detection of planted high-influence points (large DFFITS,
  influential masks flag the planted indices, high_leverage_points
  catches extreme x values, custom thresholds tighten/loosen masks).
  tests/rust_api.rs: smoke tests in diagnostic_fits for all four
  *_fit entry points.

Full Python suite: 448/448 pass (441 prior + 7 new).
Rust integration: 12/12 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented May 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sipemu
sipemu merged commit dac75f8 into main May 28, 2026
18 checks passed
@sipemu
sipemu deleted the feature/issue-27-influence branch May 28, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant