merge: synchronize benchmark dashboard with statgpu 0.2.4 - #99
Conversation
…-0.2.5 fix: harden runtime and estimator contracts
release: prepare statgpu 0.2.4
…0.2.4 docs: rebuild development roadmap for 0.2.4
8b57a65 to
e2c5519
Compare
d51de45 to
5f50754
Compare
7ca0187 to
c394acc
Compare
TheHiddenObserver
left a comment
There was a problem hiding this comment.
Final fresh review — no new in-scope findings
Reviewed the final head c394accf713d798ef8adf6474a75ebeb987e8ce1 after the complete review-fix cycle.
Merge topology and scope
- The synchronization merge commit
8c72e94e3c8b1a1527574ce1fe3b220acb09c562has the required parents:- PR #76 head
10874915f05dcae8f0124870e7d10de31848d42b; - current synchronized master
f95447e8e1bf189f16ccc2d113b3acab2ae53dfe.
- PR #76 head
- Subsequent commits only refresh deterministic generated assets and synchronize the bilingual documentation indexes.
- No new benchmark source registration, parser family, schema version, model family, CV source, or frontend feature was introduced.
- All temporary synchronization and validation workflows are absent from the final tree.
Conflict resolutions
Rechecked the five overlapping paths:
.gitignoreretains current master protections and the reviewed dashboard source/artifact exceptions;pyproject.tomlremains at statgpu 0.2.4 and only addsjsonschema[format]>=4.0to the relevant extras;- English and Chinese benchmark guides retain the reviewed PR #76 dashboard contract without restoring stale universal claims;
- English and Chinese documentation indexes now expose consistent dashboard navigation and carry the current update date.
Canonical data and provenance
- manifest minimum source date remains
2026-06-01; - 8 sources remain registered, available, and parsed;
- 0 sources are skipped;
- 1,774 normalized runs and 36 models remain unchanged;
- inherited unregistered master evidence increases catalog coverage to 40 but does not create dashboard rows;
- generation ID is consistent across benchmark data, parse report, and source inventory;
- frontend and deployed documentation data copies are deterministic and current;
- no measured row was fabricated or inferred.
Validation
- Benchmark Frontend CI #355: success, including Python 3.9/3.11 data tests, strict generation/schema validation, TypeScript typecheck, Vite build, staleness, and Playwright Chromium E2E.
- Exact-head project validation run 31101612155: success, covering documentation/static contracts, Python 3.9–3.12 regression matrix, full CPU suite, and sklearn 1.2.2/1.3.2/latest compatibility.
- Exact-head documentation validation run 31102102289: success after the bilingual navigation fix.
Review conclusion
No unresolved correctness, reproducibility, provenance, merge-topology, scope, documentation, or CI finding remains in this synchronization PR. The latest fresh review found no new in-scope issue. The PR is ready for review/merge into the PR #76 staging branch, but it should not be merged into master directly.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
TheHiddenObserver
left a comment
There was a problem hiding this comment.
Additional data-completeness review
The synchronization mechanics and deterministic bundle remain valid, but a fresh review focused on data completeness found an important limitation that should qualify the earlier “no new in-scope findings” conclusion.
1. The bundle is internally complete, not method-coverage complete
files_skipped=0 means that all eight registered sources were present and parsed. It does not mean that every implemented/runnable method has current benchmark data.
The repository’s own coverage audits explicitly classify the following as benchmark-data gaps or non-canonical-ready evidence:
- all current CV families (
RidgeCV,LassoCV,ElasticNetCV,LogisticRegressionCV,PenalizedGLM_CV,CoxPHCV); - Feature Selection;
- Bisquare/Fair and all-backend robust/quantile coverage;
- systematic linear/GLM inference;
- penalized CoxPH and CoxPHCV;
- most covariance estimators;
- substantial nonparametric breadth;
- additional panel estimators;
- multiple-testing/resampling utilities;
- distribution benchmarks, where a June Markdown result exists but no canonical structured source exists;
- broader Ordered and ANOVA crossover grids.
Issue #91 is the specific open issue tracking the missing canonical CV source. It correctly states that the dashboard has a CV presentation contract but no real current CV rows.
2. source_inventory.json does not prove source-catalog completeness
The current implementation sets:
eligible_total = registered_sourcesrather than independently classifying all discovered artifacts against the minimum-date, provenance, structure, and alignment rules.
Likewise, when the manifest does not provide catalog_total, the generator computes it by recursively counting JSON files outside benchmark_frontend_sources; it is not a reviewed catalog of candidate sources. The repository has no committed benchmark_source_catalog.json, and frontend_data/catalog.py still describes itself as a placeholder.
Therefore the current values:
catalog_total = 40
eligible_total = 8
registered_sources = 8
available_sources = 8
parsed_sources = 8
must be interpreted as:
- 40 discovered non-canonical JSON artifacts under the counting rule;
- 8 manifest-registered sources;
- all 8 registered sources are available and parsed.
They do not establish that only eight repository artifacts are genuinely eligible or that no benchmark data is missing.
3. Historical missing-input finding
PR #76 previously had a separate concrete defect where the registry referenced benchmark inputs that were not committed, causing clean-checkout run-count drift and parser-test failure. The current strict manifest/SHA/staleness gates address that class of defect. That historical finding is different from the present method-coverage gaps.
Review conclusion
- This does not invalidate PR #99 as a synchronization-only PR; #90 explicitly excludes new benchmark families.
- It does invalidate any interpretation that the dashboard is data-complete merely because
8/8/8sources parse and 1,774 rows are generated. - Before PR #76 is proposed for integration into
master, its body/docs should include a concise Known data gaps section linking #91 and the coverage-audit documents. - A separate umbrella issue should be considered for source-inventory/catalog semantics and non-CV coverage gaps, because #91 covers CV only.
No additional synchronization-code defect was found in this pass, but the provenance/completeness wording should be tightened.
TheHiddenObserver
left a comment
There was a problem hiding this comment.
Final pre-merge review — ready to merge into PR #76 staging branch
Reviewed the current final head c394accf713d798ef8adf6474a75ebeb987e8ce1 again immediately before merge.
Findings
No new blocking correctness, synchronization, provenance, generated-asset, documentation, or CI issue was found.
- GitHub reports PR #99 open, non-draft, and mergeable.
- The base branch is still the unchanged PR #76 head
10874915f05dcae8f0124870e7d10de31848d42b. - The synchronization head is still
c394accf713d798ef8adf6474a75ebeb987e8ce1. - There are no unresolved inline review threads.
- Benchmark Frontend CI #353 and #355 both completed successfully on the final head.
- Exact-head project validation run
31101612155completed successfully, covering the Python 3.9–3.12 regression matrix, full CPU suite, static/docs contracts, and sklearn compatibility. - The 482-file PR view is explained by importing 1,242 commits of current
masterhistory into the old dashboard staging branch. Relative to synchronized master, the net tree remains the dashboard pipeline/frontend/tests/docs/source bundle plus the reviewed configuration changes. - No temporary synchronization or validation workflow is present in the final changed-file inventory.
- Registered-source integrity remains 8 registered / 8 available / 8 parsed / 0 skipped, with 1,774 runs and 36 represented models. This is correctly disclosed as registered-bundle integrity rather than complete method coverage; #100 and #91 own the remaining catalog/coverage work.
Required merge method
Use Create a merge commit. Do not squash or rebase this stacked synchronization PR: those methods would collapse or replay the imported master history and defeat the reviewed two-parent synchronization topology.
Post-merge gate
After merging PR #99 into feature/benchmark-frontend-dashboard, re-fetch PR #76, confirm its head moved to the merge result, check mergeability against current master, and run/review the resulting PR #76 checks before closing #90.
Conclusion
PR #99 is ready to merge into the PR #76 staging branch. It must not be merged directly into master. The lack of complete benchmark-family coverage is documented follow-up work, not a blocker for this synchronization-only merge.
565be3a
into
feature/benchmark-frontend-dashboard
Summary
Synchronize the benchmark dashboard staging branch (PR #76) with current
masterwithout adding a new benchmark source, parser family, schema field, model family, CV source, or frontend feature.This is a stacked synchronization PR:
feature/benchmark-frontend-dashboard;c394accf713d798ef8adf6474a75ebeb987e8ce1;10874915f05dcae8f0124870e7d10de31848d42b;f95447e8e1bf189f16ccc2d113b3acab2ae53dfe;8c72e94e3c8b1a1527574ce1fe3b220acb09c562;5f50754026448a0723fbef7d1ed6515465ec9a7a;c394accf713d798ef8adf6474a75ebeb987e8ce1.Reviewed merge resolution
Five paths changed on both sides:
.gitignore;docs/en/README.md;docs/en/guides/benchmarks.md;docs/cn/guides/benchmarks.md;pyproject.toml.Git auto-merged
.gitignoreand the English documentation index; the two benchmark guides andpyproject.tomlproduced textual conflicts. The final resolution was reconstructed explicitly from the source commits:jsonschema[format]>=4.0, navigation, and tracked-source exceptions are reapplied;Registered-bundle integrity
Canonical registered bundle after deterministic regeneration:
1.1.0;2026-06-01;7437ea045e8161b4090598bdfb8062068514a5914f5d7e10a0bb54fc8dbbeff1.These figures establish integrity of the manifest-registered source set. They do not establish complete benchmark coverage across maintained statgpu capabilities.
Inventory semantics and data-completeness limits
Current master adds Cox/PR79/PR80 benchmark evidence under
results/benchmark_frontend_sources/. Those files are preserved as inherited master history but are intentionally not registered infrontend_sources.jsonand do not create dashboard rows in this synchronization PR.The generated inventory currently reports
catalog_total=40, but this value is a raw JSON-artifact count under the current scan rule, not an audited classification of 40 benchmark candidates. Likewise,eligible_total=8currently aliases the manifest registration count rather than an independent eligibility audit.Accordingly:
40must not be described as complete catalog coverage;8/8/8means every registered source exists and parses;The audited source catalog, corrected inventory semantics, and maintained method-coverage matrix are tracked in #100. The first dedicated missing-source package, current cross-validation data, is tracked in #91.
No measured row is fabricated or inferred.
Known benchmark-data gaps
The synchronized bundle still lacks or only partially covers substantial maintained capability areas, including:
Some existing evidence is not canonical-ready because it is rounded Markdown, lacks raw provenance/timing dimensions, has incomplete scale identity, or has known alignment limitations. #100 owns classification and issue assignment for these cases.
Merge topology
The synchronization was produced by a real two-parent merge. Temporary one-shot synchronization and validation workflows were removed from the final history/tree. The merge parents are exactly:
10874915f05dcae8f0124870e7d10de31848d42b— PR feat: benchmark dashboard frontend and canonical data pipeline #76 head;f95447e8e1bf189f16ccc2d113b3acab2ae53dfe— synchronized master.Validation
Dashboard-specific final head
Benchmark Frontend CI #355 / run
31102163926: successProject-level exact-head validation
Run
31101612155: success against exact content head5f50754026448a0723fbef7d1ed6515465ec9a7aThe only later changes are the two documentation-index edits in
c394accf....Run
31102102289: success against exact final headc394accf713d798ef8adf6474a75ebeb987e8ce1Review-fix completion
Review/fix cycles resolved:
catalog_total,eligible_total, and registered-source success as evidence of complete data coverage.The latest review finds no remaining synchronization, reproducibility, provenance, topology, documentation, or CI defect in #90 scope. Benchmark coverage completeness remains intentionally unresolved and is now explicitly tracked by #100 and its child work packages.
Merge target
This PR is ready for review/merge into the PR #76 staging branch. It must not be merged directly into
master. Issue #90 remains open until this stacked PR is merged intofeature/benchmark-frontend-dashboardand PR #76 is confirmed mergeable.