Skip to content

Commit e2662e2

Browse files
TheHiddenObserverTheHiddenObserver
committed
docs: rebuild development roadmap for 0.2.4
1 parent 0aeeb95 commit e2662e2

5 files changed

Lines changed: 521 additions & 174 deletions

File tree

‎dev/plans/ISSUES.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# statgpu Roadmap Issue Index
2+
3+
> Last synchronized: **2026-08-06**
4+
> Roadmap PR: **#89**
5+
> Baseline release: **0.2.4**
6+
7+
This file maps the canonical roadmap to executable GitHub issues. GitHub issue state is authoritative; this file is a navigation index and must be updated when issues are split, superseded, or closed.
8+
9+
## Active issues
10+
11+
| Priority | Issue | Work package | Dependencies |
12+
|---|---:|---|---|
13+
| P0 | #90 | Synchronize benchmark dashboard PR #76 with the 0.2.4 `master` baseline | PR #89 planning reference |
14+
| P1 | #91 | Add a canonical cross-validation benchmark source and dashboard coverage | #90 |
15+
| P1 | #92 | Complete dashboard production QA, cross-browser smoke, accessibility, and documentation integration | #90; preferably #91 before final QA |
16+
| P1 | #93 | Complete Panel Tier-1 shared framework, diagnostics, fit statistics, and covariance support | Independent of dashboard lane |
17+
| P2 | #94 | Implement Kaplan-Meier and Nelson-Aalen estimators | Independent; shares survival result design with future work |
18+
| P2 | #95 | Implement initial Weibull, log-normal, and log-logistic AFT family | May proceed independently; sequence after #94 unless resources justify parallel work |
19+
| P2 | #96 | Design and implement multinomial logistic regression Phase 1 | Independent of dashboard; sparse support deferred |
20+
| P2 | #97 | Define the shared sparse-array/backend contract with no silent densification | Prerequisite for HDFE, mixed models, and broad sparse estimator support |
21+
22+
## Recommended sequencing
23+
24+
### Product and benchmark lane
25+
26+
```text
27+
#90 → #91 → #92 → propose PR #76 for master integration
28+
```
29+
30+
Do not add new benchmark families during #90. Do not perform final dashboard QA on a stale or unsynchronized branch.
31+
32+
### Statistical workflow lane
33+
34+
```text
35+
#93
36+
#94 → #95
37+
#96
38+
#97 → future sparse estimator issues
39+
```
40+
41+
These issues may proceed in parallel only when they do not compete for the same backend, inference, or review surface.
42+
43+
## Issue maintenance rules
44+
45+
- Keep one primary issue per statistical or product contract.
46+
- Split an issue when one PR would combine independent parameterizations, estimator families, or acceptance gates.
47+
- Add explicit links when an issue blocks or is blocked by another issue.
48+
- Update `ROADMAP.md`, `TO_DO.md`, and this file when priorities change.
49+
- Close issues only with merged implementation evidence, required CI, external alignment where applicable, physical-GPU validation, and synchronized documentation.
50+
- Do not close an issue solely because a class, function, parser, or frontend control exists.

‎dev/plans/README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# statgpu Development Plans
2+
3+
This directory contains the project roadmap, execution backlog, and historical design notes.
4+
5+
## Source-of-truth hierarchy
6+
7+
Use the following order when documents disagree:
8+
9+
1. **Current public capability:** `docs/en/guides/implemented-methods.md` and the linked model pages.
10+
2. **Current development priorities:** [`ROADMAP.md`](ROADMAP.md).
11+
3. **Executable work and dependencies:** open GitHub issues, summarized in [`ISSUES.md`](ISSUES.md), and active pull requests.
12+
4. **Development gates:** `dev/AGENTS.md` and the applicable `.claude/workflows/` and `.claude/skills/` files.
13+
5. **Module plans in this directory:** research background and design context only. Their checklists are not a reliable current capability inventory unless the document states a recent verification release and commit.
14+
15+
`TO_DO.md` is retained as a compact compatibility entry point and must remain synchronized with `ROADMAP.md` and `ISSUES.md`.
16+
17+
## Verified baseline
18+
19+
- Last verified release: **statgpu 0.2.4**
20+
- Last verified commit: `0aeeb95b60e3e274053b8f1b6427ae50c8eec015`
21+
- Verification date: **2026-08-06**
22+
- Release workflow, PyPI wheel/sdist publication, clean production installation, and representative model smoke tests passed.
23+
24+
The release baseline does not imply that every historical plan item is complete. It establishes the code and documentation snapshot from which future work must branch.
25+
26+
## Document status
27+
28+
| Document | Status | How to use it |
29+
|---|---|---|
30+
| `ROADMAP.md` | Canonical | Current priorities, sequencing, dependencies, and definition of done. |
31+
| `ISSUES.md` | Canonical navigation | Maps roadmap work packages to executable GitHub issues and dependency order. GitHub issue state remains authoritative. |
32+
| `TO_DO.md` | Canonical summary | Compact queue synchronized with the roadmap and issue index. |
33+
| `panel_framework_proposal.md` | Active design reference | Shared panel architecture and Tier-1 diagnostics proposal. Validate details against current code before implementation. |
34+
| `plan_survival.md` | Active module reference | Cox Phase 1 status and Survival Phase 2+ scope; last materially updated 2026-07-12. |
35+
| `cran_r_package_mapping.md` | Comparative reference | Method-family gap map. Some individual rows may lag current implementation. |
36+
| `plan_anova.md` | Historical research plan | Its early implementation-status header is stale; use implemented-methods/model docs for current ANOVA support. |
37+
| `plan_covariance.md` | Historical research plan | Its early implementation-status header is stale; current covariance estimators are documented elsewhere. |
38+
| `plan_krr.md` | Historical research plan | Nystroem, KernelPCA, and chi-square kernel status in the old checklist is stale. |
39+
| `plan_spline.md` | Historical research plan | SplineTransformer, cyclic, and thin-plate status in the old checklist is stale. |
40+
| `plan_unsupervised.md` | Historical phase record | Useful for benchmark and algorithm history, not the current priority queue. |
41+
| `plan.md` | Historical delta | Superseded by `ROADMAP.md`. |
42+
| `archive/` | Archive | Completed or superseded planning material. |
43+
44+
## Planning rules
45+
46+
A roadmap item becomes executable only after it has a GitHub issue that defines:
47+
48+
- user or developer problem;
49+
- scope and explicit non-goals;
50+
- public API and failure behavior;
51+
- NumPy, CuPy, and Torch backend contract;
52+
- inference and formula implications where applicable;
53+
- external baselines and normalization/alignment settings;
54+
- unit, regression, compatibility, and physical-GPU validation;
55+
- documentation and benchmark deliverables;
56+
- dependencies and completion criteria.
57+
58+
Do not mark a module complete using only an implementation count or a passing CPU smoke test. Completion is contract-based and evidence-based.

‎dev/plans/ROADMAP.md‎

Lines changed: 258 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,258 @@
1+
# statgpu Roadmap
2+
3+
> Canonical development roadmap
4+
> Last verified release: **0.2.4**
5+
> Last verified commit: `0aeeb95b60e3e274053b8f1b6427ae50c8eec015`
6+
> Last verified: **2026-08-06**
7+
8+
## 1. Purpose and authority
9+
10+
This document defines the current development sequence. It is not a public support matrix.
11+
12+
- For implemented public methods and backend support, use `docs/en/guides/implemented-methods.md` and the linked model pages.
13+
- For executable work, use open GitHub issues and pull requests.
14+
- For hard development gates, use `dev/AGENTS.md` and the applicable `.claude` workflow/skill.
15+
- Module-specific plans under `dev/plans/` provide design and literature context but may contain historical checklists.
16+
17+
When a module plan conflicts with current public documentation or tests, the current documentation and validated implementation win. Update the stale plan rather than reimplementing an already delivered feature.
18+
19+
## 2. Baseline after 0.2.4
20+
21+
Version 0.2.4 established a stable correctness baseline for:
22+
23+
- public estimator validation and sklearn cloning;
24+
- transactional refits and cross-validation behavior;
25+
- solver/penalty compatibility and narrow numerical fallbacks;
26+
- analytic-weight semantics;
27+
- NumPy/CuPy/Torch finite-input handling;
28+
- binary LogisticRegression and GLM correctness;
29+
- CoxPH/CoxPHCV core contracts;
30+
- release packaging and production installation.
31+
32+
The next cycle should convert that correctness baseline into maintainable product evidence and complete selected statistical workflows. It should not immediately expand into many unrelated zero-percent modules.
33+
34+
## 3. Prioritization principles
35+
36+
Work is ranked by the following criteria:
37+
38+
1. **Correctness and contract risk:** fix ambiguous or incomplete public behavior before adding breadth.
39+
2. **Workflow completeness:** finish a partially implemented statistical workflow before starting a new module family.
40+
3. **Shared infrastructure leverage:** prefer work that reduces duplication or enables several later features.
41+
4. **Evidence quality:** implementation, external alignment, physical-GPU validation, benchmark provenance, and documentation must move together.
42+
5. **Controlled scope:** avoid PRs that combine framework refactors, multiple new model families, and broad performance work.
43+
44+
## 4. Current priority queue
45+
46+
### P0 — Roadmap and integration control
47+
48+
#### P0.1 Reconcile planning documents with 0.2.4
49+
50+
Deliverables:
51+
52+
- establish this file as the canonical roadmap;
53+
- keep `TO_DO.md` synchronized as a compact queue;
54+
- classify older module plans as active references, historical research plans, or archive material;
55+
- create GitHub issues for every active work package;
56+
- require future roadmap changes to cite an implementation, test, release, or issue.
57+
58+
#### P0.2 Synchronize benchmark dashboard PR #76 with current `master`
59+
60+
PR #76 is the only active product branch at the 0.2.4 baseline, but it was built from an older base and is not currently mergeable.
61+
62+
The synchronization change must be isolated from new benchmark families:
63+
64+
- merge or rebase current `master` into the dashboard branch;
65+
- resolve test, workflow, documentation, package-layout, and generated-asset conflicts;
66+
- regenerate the deterministic three-file data bundle and deployment assets;
67+
- rerun Python, TypeScript, build, staleness, and Playwright gates;
68+
- preserve source hashes, canonical identities, and no-fabrication rules.
69+
70+
### P1 — Benchmark evidence and dashboard readiness
71+
72+
#### P1.1 Add a canonical cross-validation benchmark source
73+
74+
The dashboard implements the CV presentation contract but has no current canonical CV source.
75+
76+
Initial matrix:
77+
78+
- `RidgeCV`;
79+
- `LassoCV`;
80+
- `ElasticNetCV`;
81+
- `LogisticRegressionCV`;
82+
- `PenalizedGLM_CV`;
83+
- `CoxPHCV`.
84+
85+
Required dimensions include backend, folds, candidate-grid size, path/warm-start configuration, CV time, final-refit time, selected parameter, score, convergence/failure diagnostics, timing scope, synchronization policy, and peak memory where available.
86+
87+
#### P1.2 Complete dashboard product QA
88+
89+
Before PR #76 is proposed for integration into `master`:
90+
91+
- test the production build from the nested documentation path;
92+
- complete Chrome/Chromium, Firefox, and WebKit/Safari smoke coverage;
93+
- verify filter cascades, chart/table consistency, empty states, and source metadata;
94+
- verify keyboard navigation, visible focus, control labels, and an accessible table path;
95+
- integrate the user guide into documentation navigation;
96+
- keep generated data and deployment assets deterministic and current.
97+
98+
URL-persisted state, mobile redesign, virtualization, and bundle partitioning remain deferred until supported by measured product need.
99+
100+
### P1 — Panel workflow completion
101+
102+
Panel data has substantial estimator coverage but lacks several standard econometric diagnostics and shared infrastructure.
103+
104+
Implement in three bounded changes:
105+
106+
1. **Shared panel base and covariance registry**
107+
- consolidate validation, fitted-state handling, summary construction, and covariance dispatch;
108+
- preserve all current numerical behavior with golden regression tests.
109+
2. **Specification tests and fit statistics**
110+
- Hausman FE-vs-RE test;
111+
- pooling F-test;
112+
- Breusch-Pagan LM test;
113+
- within, between, overall, and adjusted R-squared;
114+
- model F-statistic;
115+
- shared structured test-result object.
116+
3. **Extended covariance support**
117+
- robust covariance for RandomEffects;
118+
- HC0/HC2/HC3 where statistically defined;
119+
- Driscoll-Kraay covariance;
120+
- explicit one-way/two-way cluster and bandwidth/kernel contracts.
121+
122+
External alignment should use `linearmodels`, R `plm`, and R/Python sandwich implementations with explicitly matched formulas, effects, covariance definitions, and degrees-of-freedom corrections.
123+
124+
Panel IV, high-dimensional fixed-effect absorption, DID/event-study, and dynamic-panel GMM are blocked on this shared foundation.
125+
126+
### P2 — Survival Phase 2
127+
128+
Cox Phase 1 is implemented. The next survival work should complete foundational analysis and prediction before advanced latent-event structures.
129+
130+
#### P2.1 Nonparametric survival estimators
131+
132+
Implement Kaplan-Meier and Nelson-Aalen with:
133+
134+
- right censoring;
135+
- backend-consistent input validation;
136+
- Greenwood or corresponding variance;
137+
- confidence intervals and median survival where defined;
138+
- stratified/grouped output;
139+
- explicit left-truncation follow-up scope;
140+
- alignment with R `survival` and `lifelines`.
141+
142+
#### P2.2 Parametric AFT models
143+
144+
Initial distributions:
145+
146+
- Weibull;
147+
- log-normal;
148+
- log-logistic.
149+
150+
Required contracts:
151+
152+
- censored likelihood and parameterization documented explicitly;
153+
- NumPy, CuPy, and Torch paths;
154+
- model-based covariance and summary output;
155+
- survival, hazard, cumulative-hazard, and quantile prediction;
156+
- formula support;
157+
- alignment with R `survreg` and `lifelines`, including scale/sign mappings.
158+
159+
Frailty, Fine-Gray competing risks, multi-state models, joint models, and survival forests remain deferred until these foundations are complete.
160+
161+
### P2 — Linear-model API parity and sparse infrastructure
162+
163+
#### P2.3 Multinomial logistic regression
164+
165+
Begin with a design issue that fixes:
166+
167+
- identifiability convention;
168+
- coefficient and probability shapes;
169+
- class and sample weighting;
170+
- likelihood and information criteria;
171+
- solver support;
172+
- penalized and CV staging;
173+
- inference scope;
174+
- sklearn compatibility.
175+
176+
The first implementation should prefer unpenalized and L2 support with fit, decision function, probability prediction, hard prediction, and model-based inference. L1/nonconvex penalties and full CV expansion should be separate follow-ups.
177+
178+
#### P2.4 Sparse backend contract
179+
180+
Define a shared sparse-input policy before adding estimator-specific support:
181+
182+
- SciPy CSR/CSC;
183+
- CuPy sparse;
184+
- Torch sparse CSR where viable;
185+
- supported operations and solver matrix;
186+
- no silent densification;
187+
- memory-budget and failure tests;
188+
- explicit unsupported combinations.
189+
190+
This work is a prerequisite for high-dimensional fixed effects, mixed models, and several large-scale algorithms.
191+
192+
### P3 — Feature-driven technical debt
193+
194+
Refactor only when a bounded feature or correctness task provides regression coverage.
195+
196+
Current candidates:
197+
198+
- split candidate generation, fold execution, selection, and final refit in `_penalized_cv.py`;
199+
- split long FISTA/FISTA-BB solver functions by state update, line search, stopping, and diagnostics;
200+
- unify repeated backend array-copy and scalar-extraction helpers;
201+
- reduce duplicated CPU/CuPy/Torch fit paths where one backend-generic implementation preserves device semantics;
202+
- unify duplicated IRLS coordinate-descent implementations only after objective and stopping contracts are frozen.
203+
204+
Do not open a single repository-wide “unify all backends and solvers” PR.
205+
206+
### P4 — Deferred module expansion
207+
208+
The following remain valid long-term directions but are not in the immediate queue:
209+
210+
- mixed-effects models and GEE;
211+
- meta-analysis;
212+
- changepoint detection;
213+
- multivariate methods;
214+
- copulas;
215+
- multiple imputation;
216+
- nonlinear least squares;
217+
- advanced ANOVA/repeated-measures workflows;
218+
- advanced robust covariance;
219+
- tensor/adaptive/shape-constrained GAM;
220+
- kernel SVM and broad unsupervised expansion.
221+
222+
A deferred module can be promoted only with a concrete user need, a scoped design, three-backend feasibility, external baselines, and a clear maintenance owner.
223+
224+
## 5. Definition of done
225+
226+
A statistical feature is complete only when all applicable items pass:
227+
228+
- public API and failure behavior are documented;
229+
- NumPy, CuPy, and Torch execution paths exist, or an explicitly approved exception is recorded;
230+
- explicit device requests do not silently fall back;
231+
- strict inference is implemented or the estimator is explicitly estimation-only;
232+
- formula semantics are tested where the API supports formulas;
233+
- external comparisons use aligned objective normalization, penalties, solvers, ties, tolerances, and feature sets;
234+
- CPU unit/regression/compatibility tests pass;
235+
- physical-GPU validation covers maintained CuPy and Torch paths;
236+
- performance claims use synchronized, provenance-bearing artifacts;
237+
- English and Chinese user documentation and changelog claims remain consistent;
238+
- no stale fitted state, hidden fallback, or untracked diagnostic script is introduced.
239+
240+
## 6. Issue hygiene
241+
242+
Each active roadmap package must have one primary GitHub issue. Split implementation into child or follow-up issues when one change would mix independent statistical contracts.
243+
244+
Every issue must include:
245+
246+
- context and user impact;
247+
- scope and non-goals;
248+
- public API decisions;
249+
- statistical definitions and parameterization;
250+
- backend/device behavior;
251+
- inference and formula implications;
252+
- external baseline matrix;
253+
- test and physical-GPU gates;
254+
- documentation and benchmark outputs;
255+
- dependencies;
256+
- acceptance criteria.
257+
258+
Close issues using evidence from merged commits, CI, external comparisons, and physical-GPU runs. Do not close an issue solely because a class or function name exists.

0 commit comments

Comments
 (0)