merge: synchronize benchmark dashboard with statgpu 0.2.4

.github/releases/v0.2.3.md

@@ -0,0 +1,155 @@
+# statgpu 0.2.3
+
+statgpu 0.2.3 is a substantial survival-analysis release. It completes the first major Cox proportional-hazards implementation phase, expands Cox model selection and penalized estimation, and hardens numerical, inference, and packaging behavior across the supported NumPy, CuPy, and PyTorch backends.
+
+## Highlights
+
+- Complete CoxPH support for **Breslow, Efron, and Exact ties**.
+- Delayed entry and `(start, stop]` counting-process data.
+- Shared-coefficient stratified Cox models with stratum-specific baselines.
+- Subject-aware repeated-row handling and `Surv(start, stop, event)` formula input.
+- Extended `CoxPHCV` scoring, fold construction, diagnostics, and final refitting.
+- Hardened L1, L2, Elastic Net, SCAD, and MCP penalized Cox estimation.
+- Universal `py3-none-any` wheel validated on Linux, Windows, and macOS.
+
+## CoxPH Phase 1 completion
+
+### Tie handling and risk sets
+
+`CoxPH` now supports all three primary tie methods:
+
+- `ties="breslow"`;
+- `ties="efron"`;
+- `ties="exact"`.
+
+The implementation uses shared NumPy, CuPy, and Torch-CUDA risk-set primitives for the partial-likelihood objective, gradient, information matrix, and baseline estimation. Exact tied-event partitions use backend-native dynamic programming rather than a CPU-only implementation.
+
+### Delayed entry, start-stop rows, and stratification
+
+The public Cox interface now supports:
+
+- delayed entry;
+- `(start, stop]` counting-process rows;
+- repeated rows belonging to the same subject;
+- shared coefficients with stratum-specific baseline hazards;
+- formula input through `Surv(start, stop, event)`.
+
+Formula-driven NA removal now keeps entry, cluster, strata, subject, response, and design arrays aligned.
+
+### Inference and baseline prediction
+
+For Breslow and Efron fits, model-based, HC0, HC1, and cluster covariance are supported where the requested data configuration is eligible. Exact ties currently support model-based covariance only; unsupported robust-covariance requests fail explicitly instead of silently changing behavior.
+
+Cox numerical stability has been strengthened through centered risk-set moments and log-domain baseline prediction. Singular information matrices are rejected rather than returning misleading zero standard errors.
+
+Baseline prediction requires `compute_inference=True`. The conventional Breslow baseline estimator is used after coefficient fitting, including when coefficients were fitted with Efron or Exact ties.
+
+## CoxPHCV completion
+
+`CoxPHCV` held-out partial likelihood now supports:
+
+- Breslow, Efron, and Exact ties;
+- delayed entry and start-stop data;
+- strata;
+- subject-grouped folds, so repeated rows from one subject remain in one fold;
+- device-native held-out scoring;
+- convergence- and failure-aware candidate diagnostics;
+- selected-penalty refitting on the complete dataset.
+
+Full-data cache identities, fold validation, cloneability, repeated fitting, pickling, and failed-refit state cleanup were hardened.
+
+Requested two-stage or successive-halving controls currently execute one deterministic exhaustive full-precision candidate pass. This avoids repeated complete-grid fitting without pretending that unsafe screening has occurred.
+
+## Penalized Cox and grouped penalties
+
+Penalized Cox estimation was hardened for:
+
+- L1;
+- L2;
+- Elastic Net;
+- SCAD;
+- MCP.
+
+The unidentified Cox intercept was removed, Cox-specific SCAD/MCP warm starts were corrected, and Torch Efron value, gradient, and Hessian paths remain native rather than routing through CuPy.
+
+`PenalizedCoxPHModel` remains an estimation-only API in this release. Passing `compute_inference=True` raises `NotImplementedError` explicitly.
+
+Public Group Lasso and Adaptive Group Lasso behavior now follows the generic loss-gradient and exact group-proximal implementation consistently across supported backends.
+
+## Reliability and API hardening
+
+This release also improves:
+
+- failed-fit and failed-refit state resets;
+- singular-information handling;
+- censoring- and tie-correct concordance semantics;
+- device-label grouping without accidental coercion;
+- independence from optional statsmodels for robust Cox covariance;
+- CV cache identity and candidate eligibility;
+- one-shot `cv_splits` iterator reuse across repeated fit, clone, reconstruction, and pickle.
+
+## Installation and platform support
+
+Base CPU installation:
+
+```bash
+pip install statgpu==0.2.3
+```
+
+CUDA extras:
+
+```bash
+pip install "statgpu[gpu11]==0.2.3"
+pip install "statgpu[gpu12]==0.2.3"
+```
+
+PyTorch backend:
+
+```bash
+pip install "statgpu[torch]==0.2.3"
+```
+
+The published wheel is a pure-Python `py3-none-any` artifact. The exact release wheel is clean-installed and exercised on Ubuntu, Windows, and macOS using Python 3.11, including a CPU `LinearRegression.fit/predict` smoke test and public Cox imports. The broader regression suite covers Python 3.9–3.12 on Ubuntu.
+
+The cross-platform wheel validation establishes CPU-wheel portability. It does not add Apple MPS support. CUDA execution still requires a compatible NVIDIA driver/runtime and the matching CuPy or PyTorch package.
+
+Optional Cython CPU accelerators are not embedded in the universal wheel. Their `.pyx`/`.pxd` sources remain in the source distribution for local builds.
+
+## Validation
+
+The final reviewed implementation head for the CoxPH Phase 1 work is:
+
+```text
+f05a44ad363b46612e956e137e2f00d040765acb
+```
+
+Hosted workflow #960 passed documentation, static, full CPU, and Python 3.9–3.12 regression jobs. The complete CPU suite reported **1881 passed and 662 skipped**.
+
+The exact-head physical-GPU promotion artifact records:
+
+- schema 3;
+- 134/134 checks passed;
+- zero child and nested return codes;
+- empty gate-failure arrays;
+- clean source state before and after execution;
+- SHA-256 `bd4058450def691dd29e9d78853534016c6da70c33192a97dc312d95cbe5d76d`.
+
+Physical-GPU evidence: https://gist.github.com/TheHiddenObserver/afdcad86a243e68a918d852b92e984a4
+
+No universal GPU speedup claim is made. Performance depends on problem size, backend, hardware, tie method, and synchronization costs.
+
+## Upgrade notes and known limits
+
+- Python 3.9 or newer is required.
+- Exact ties do not currently provide robust or cluster covariance.
+- `PenalizedCoxPHModel` inference is not implemented in 0.2.3.
+- Baseline prediction requires inference-enabled fitting.
+- Two-stage and successive-halving CoxPHCV controls currently use the documented exhaustive single-pass safety strategy.
+- Apple MPS is not currently a statgpu device backend.
+
+## Full change history
+
+- Main implementation: https://github.com/TheHiddenObserver/statgpu/pull/80
+- Release preparation: https://github.com/TheHiddenObserver/statgpu/pull/86
+- Full comparison: https://github.com/TheHiddenObserver/statgpu/compare/v0.2.2...v0.2.3
+- Repository changelog: https://github.com/TheHiddenObserver/statgpu/blob/master/CHANGELOG.md

.github/releases/v0.2.4.md

@@ -0,0 +1,113 @@
+# statgpu 0.2.4
+
+statgpu 0.2.4 is a maintenance and correctness release focused on reliable estimator contracts across NumPy, CuPy, and PyTorch. It hardens generalized linear models, direct and cross-validated linear-model wrappers, solver dispatch, analytic-weight handling, finite-input validation, and Torch compilation behavior. The release does not introduce a new model family; it makes existing public APIs safer, more consistent, and easier to diagnose.
+
+## Highlights
+
+- Corrected arbitrary-link Binomial IRLS and direct `LogisticRegression` weighting, likelihood, convergence, and prediction contracts.
+- Made `RidgeCV`, `ElasticNetCV`, `LogisticRegressionCV`, and unified penalized cross-validation failure-safe and backend-consistent.
+- Added backend-native finite-value, shape, response-domain, and analytic-weight validation without unnecessary full GPU-to-CPU copies.
+- Removed silent or over-broad numerical fallbacks that could hide CUDA OOM, device, indexing, contract, or programming errors.
+- Corrected solver/penalty compatibility so smooth solvers reject non-smooth objectives instead of optimizing only part of the declared problem.
+- Kept internal Torch execution eager by default while preserving explicit, observable `torch.compile` opt-in modes.
+- Improved scikit-learn cloning, tags, nested `set_params`, transactional refits, formula alignment, and fresh-interpreter import stability.
+
+## Logistic regression and GLM correctness
+
+Binomial IRLS now uses the correct Fisher weights, working response, and Bernoulli line-search objective for arbitrary supported links. Warm starts, analytic weights, and quadratic penalties are normalized and validated on the selected backend, device, and dtype before numerical work.
+
+Direct `LogisticRegression` now enforces strict binary responses and finite controls, clears stale fitted state before every refit, reports non-convergence explicitly, and uses the registered numerically stable logistic objective for fitted log-likelihood diagnostics across NumPy, CuPy, and Torch. Likelihood, AIC, BIC, pseudo-R², and convergence diagnostics remain available independently of covariance inference.
+
+Hard predictions use integer dtype on every backend. Single-column responses no longer trigger accidental broadcasting, and non-finite decision thresholds are rejected. Confusion-matrix metrics remain available for one-class targets, while ROC-AUC and average precision retain their explicit class-support requirements. Analytic weights stay device-native on CuPy and Torch paths rather than being copied wholesale to NumPy solely for CPU inference bookkeeping.
+
+GLM sample weights now follow one analytic-weight convention across fitting, ridge scaling, line search, pseudo-loglikelihood, information criteria, dispersion, and sandwich covariance. Globally rescaling analytic weights does not change fitted parameters or reported diagnostics. Formula sample weights are aligned only after Patsy determines the retained rows, including device-native Torch and CuPy alignment.
+
+## Cross-validation and estimator behavior
+
+Dedicated `RidgeCV`, `ElasticNetCV`, and `LogisticRegressionCV` fits are transactional: every fit attempt clears stale state, candidate selections are not published until the final full-data refit succeeds, and `device="auto"` keeps the backend selected during cross-validation for the final refit.
+
+Default Logistic and Elastic Net regularization grids incorporate analytic weights and satisfy integer-weight row-replication equivalence. Validation scoring preserves the estimator's declared loss instead of silently substituting MSE for non-Gaussian objectives. Optional optimized scoring and Lipschitz recovery are narrow and visible; programming errors, shape errors, CUDA OOM, and device failures remain fatal.
+
+`ElasticNet` and `ElasticNetCV` now expose the maintained post-fit inference contract directly. With `compute_inference=True`, inference is run only after the selected penalty parameters are refit on the full dataset; fold models remain estimation-only. The public documentation also reflects the shared average-loss scaling under which `ElasticNet(alpha, l1_ratio=0)` matches `Ridge(alpha)`.
+
+Estimator constructor values are retained separately from normalized runtime attributes so legacy and current scikit-learn clone checks work. Nested `set_params`, fitted-state invalidation, public tags, and finite-input wrappers follow transactional behavior.
+
+## Solver and backend safety
+
+The solver matrix now treats Elastic Net and other proximal penalties as non-smooth. Newton, L-BFGS, and L-BFGS-B reject unsupported non-smooth combinations instead of optimizing only the smooth portion of the objective. The previous Euclidean-prox Newton shortcut was removed because it did not solve the required Hessian-metric proximal subproblem. Direct non-smooth proximal-Newton requests now emit a visible warning and use backend-native FISTA.
+
+Newton-family Armijo backtracking suppresses only recognized numeric-domain trial failures. Linear solves fall back to least squares only for genuine rank failures. CUDA OOM, device, index, input-contract, and unrelated runtime failures propagate to the caller instead of being converted into misleading numerical recovery.
+
+Warm starts for FISTA, Newton-family methods, L-BFGS-family methods, and ADMM follow the preprocessed design backend, device, and dtype. ADMM's legitimate Cholesky fallback is initialized correctly, and L-BFGS-B preserves feasible directions and backend-native bounds.
+
+The public import surface is also more robust. `CoxPartialLikelihoodLoss` is exposed lazily, removing a package-initialization cycle between `statgpu.glm_core`, survival losses, and linear-model imports. GLM internals and `LogisticRegression` can now be imported in either order in a fresh interpreter.
+
+## Torch compile policy
+
+Internal iterative Torch kernels remain eager when `STATGPU_TORCH_COMPILE_MODE` is unset, `auto`, or `disable`. Compilation is enabled only when users explicitly select `default` or `reduce-overhead`.
+
+Compile construction and runtime decisions remain observable through diagnostics. Only the known CUDA Graph overwritten-output lifecycle failure becomes a permanent eager fallback for the affected callable; unrelated runtime failures remain visible. Benchmarks on the tested RTX 4090 workload did not establish a universal end-to-end speedup, so this release makes no fixed GPU acceleration or compile-performance claim.
+
+## Installation and platform support
+
+Base CPU installation:
+
+```bash
+pip install statgpu==0.2.4
+```
+
+CUDA extras:
+
+```bash
+pip install "statgpu[gpu11]==0.2.4"
+pip install "statgpu[gpu12]==0.2.4"
+```
+
+PyTorch backend:
+
+```bash
+pip install "statgpu[torch]==0.2.4"
+```
+
+The official wheel remains a pure-Python `py3-none-any` artifact built with `STATGPU_NO_EXT=1`. Release-package validation builds and checks the wheel and source distribution, clean-installs the sdist on Ubuntu, and installs the same wheel artifact in fresh Ubuntu, Windows, and macOS environments. Optional Cython sources remain available in the sdist for local builds.
+
+Cross-platform CPU-wheel validation does not add Apple MPS support. CUDA execution still requires a compatible NVIDIA driver/runtime and the matching CuPy or PyTorch package.
+
+## Validation
+
+The implementation delivered by pull request #87 completed repeated review-fix cycles with no unresolved critical, high, or in-scope medium findings.
+
+Hosted validation on the final implementation head reported:
+
+- complete CPU suite: 2239 passed and 719 skipped;
+- static and documentation contracts: passed;
+- Python 3.9, 3.10, 3.11, and 3.12 regression jobs: passed;
+- scikit-learn 1.2.2, 1.3.2, and latest compatibility jobs: passed;
+- release-note and release-package validation: passed;
+- clean sdist installation and Ubuntu, Windows, and macOS wheel smoke tests: passed.
+
+Physical NVIDIA validation covered the unchanged numerical runtime:
+
+- RTX 4090 with PyTorch 2.8.0+cu128: selected Torch compile/CUDA Graph matrix passed 9/9, and LogisticRegression/IRLS runtime assertions passed;
+- Tesla P100-SXM2-16GB with CuPy 13.6.0: LogisticRegression/IRLS runtime assertions passed.
+
+The focused 0.2.4 release pull request changes version metadata, changelog organization, and release documentation only. The final release tag must be created from the validated release-PR merge commit according to `RELEASING.md`.
+
+## Upgrade notes and known limits
+
+- Code that relied on unsupported smooth-solver/non-smooth-penalty combinations may now receive an explicit error or visible FISTA delegation instead of a silently incomplete optimization.
+- Hard logistic predictions are integer-valued; downstream code should not rely on floating label dtype.
+- Invalid or non-finite thresholds and malformed scalar GLM responses now fail before solver or fold dispatch.
+- `device="auto"` cross-validation paths preserve the selected backend for final refitting, which may differ from prior accidental backend drift.
+- Torch compilation is opt-in. Unset and `auto` modes remain eager, and performance must be benchmarked on the target workload.
+- Exact-tie Cox robust or cluster covariance and `PenalizedCoxPHModel` inference remain unsupported as documented for 0.2.3.
+- Apple MPS is not a supported statgpu device backend.
+- PyPI artifacts are immutable; a publication failure after partial upload requires a new patch version rather than replacing files.
+
+## Full change history
+
+- Main implementation and review-fix work: https://github.com/TheHiddenObserver/statgpu/pull/87
+- Release preparation: https://github.com/TheHiddenObserver/statgpu/pull/88
+- Comparison with the previous release: https://github.com/TheHiddenObserver/statgpu/compare/v0.2.3...v0.2.4
+- Repository changelog: https://github.com/TheHiddenObserver/statgpu/blob/master/CHANGELOG.md
+- Release procedure: https://github.com/TheHiddenObserver/statgpu/blob/master/RELEASING.md

.github/workflows/maintenance-compatibility.yml

@@ -0,0 +1,45 @@
+name: Maintenance compatibility
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+permissions:
+  contents: read
+
+jobs:
+  sklearn-compatibility:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        sklearn_spec:
+          - scikit-learn==1.2.2
+          - scikit-learn==1.3.2
+          - scikit-learn
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install compatibility environment
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install "numpy<2" scipy pytest packaging pandas patsy "${{ matrix.sklearn_spec }}"
+          python -m pip install -e . --no-deps
+
+      - name: Validate maintenance benchmark syntax
+        run: |
+          python -m py_compile dev/benchmarks/benchmark_torch_compile_maintenance.py
+
+      - name: Run maintenance regressions
+        run: |
+          python -m pytest \
+            dev/tests/test_maintenance_024_025.py \
+            dev/tests/test_legacy_sklearn_integration.py \
+            dev/tests/test_second_full_review.py::TestEstimatorCloneAndFeatureSelectionBackend::test_all_default_public_estimators_clone \
+            -q --tb=short