Skip to content

feat: add Panel Tier-1 Stage B diagnostics - #122

Merged
TheHiddenObserver merged 165 commits into
masterfrom
agent/panel-p1-stage-b-diagnostics
Aug 9, 2026
Merged

feat: add Panel Tier-1 Stage B diagnostics#122
TheHiddenObserver merged 165 commits into
masterfrom
agent/panel-p1-stage-b-diagnostics

Conversation

@TheHiddenObserver

@TheHiddenObserver TheHiddenObserver commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

Tracks #93 and implements Stage B / PR-B of the Panel Tier-1 roadmap on top of merged Stage A (#119).

Stage-B scope includes Hausman FE-vs-RE, pooling F, Breusch-Pagan LM, parameter-based panel R², adjusted R²/model F where defined, structured diagnostics, NumPy/CuPy/Torch parity, external-definition checks, frontend evidence, and physical GPU acceptance.

Final lifecycle status

COMPLETE / ready for review.

Current PR head: cdff57179d97cd53863681cfa46301da919999ec.

The complete review/fix loop was rerun under .claude/skills/code-review.md using changed+adjacent review and auto-fix semantics. Fresh final review on this exact head found:

  • CRITICAL: 0
  • HIGH: 0
  • relevant MEDIUM: 0
  • unresolved review threads: 0

Fresh final review ID: 4891642296.

Applicable-Hausman review fix

The final P2 required the physical GPU gate to execute at least one genuinely applicable Hausman statistic/p-value/df path rather than counting only matching structured-inapplicable results.

The runner now includes a deterministic fitted FE/RE fixture with seed 20260810, 12 entities x 4 observations, one slope, nonzero entity-effect scale 0.005, noise scale 0.1, and 48 observations. Every requested GPU backend must produce at least one successful applicable=true Hausman result with finite statistic/p-value and positive df. The four existing non-PSD parameterizations remain, so both applicability branches are validated.

Accepted physical P100 evidence

Exact clean measured implementation:

2701aa9feb3796c33c94e6480fcb78c80c6a809c

Raw artifact:

results/pr122_p100/panel_stage_b_gpu_validation_2701aa9f.json

Provenance:

  • artifact commit: 0d0d654d825cea872672f27d02107a58048b345f
  • raw Git blob: fa3a253e6d882a4e69be29e7e3b1dce7b223b9a9
  • clean working tree: true
  • Tesla P100-SXM2-16GB
  • Python 3.9.16
  • NumPy 1.24.2
  • SciPy 1.10.1
  • Torch 2.0.0
  • timing collected: false

Both physical GPU backends passed:

  • CuPy: 17/17 estimator cases, 5/5 Hausman diagnostics, executed_backend=cupy, no CPU fallback;
  • Torch: 17/17 estimator cases, 5/5 Hausman diagnostics, executed_backend=torch, no CPU fallback.

The original four Hausman parameterizations remain successful structured applicable=false / non-PSD cases.

hausman_applicable_nonzero_effect is physically applicable=true on both backends with df=1:

  • CuPy statistic 1.1965942530851057, p-value 0.27400344142676447;
  • Torch statistic 1.1965942530849238, p-value 0.2740034414268009;
  • NumPy reference statistic 1.196594253085033, p-value 0.274003441426779.

Statistic differences versus NumPy are 7.26e-14 (CuPy) and 1.09e-13 (Torch); FE/RE coefficient and diagnostic-covariance differences are below 1e-12.

The historical focused disconnected two-way FE artifact remains supplementary evidence for the unchanged df=1 path and is explicitly identified with its original measurement SHA a57efcea29b0e87ecb89865c5a6902d5773812c6; it is not relabeled as a 2701aa9 measurement.

Canonical frontend evidence

Promoted source:

results/benchmark_frontend_sources/panel_stage_b_pr122_p100_20260809_2701aa9f.json

  • SHA-256: 2056f836bfe2a708b3131becca42dbba15e519762c8bafb582000b19f81120bf
  • source id: panel-stage-b-pr122-20260809-2056f836bfe2
  • validation-only rows: 44 = 17 estimator cases x 2 backends + 5 Hausman diagnostics x 2 backends
  • no timing or speedup claims
  • applicable Hausman statistic/pvalue/df are preserved; missing/invalid numeric evidence fails closed
  • previous 42-row a57efcea... source is retained as explicitly superseded historical audit evidence
  • frontend/public and docs/assets generated JSON mirrors are byte-identical.

Compare 2701aa9f...cdff5717 contains no statgpu/ numerical source change and no physical-runner change. Post-measurement changes are raw/canonical evidence, contracts, generated assets, and documentation only, so the P100 evidence remains applicable to the final candidate.

Exact-final hosted validation

For current head cdff57179d97cd53863681cfa46301da919999ec:

  • Tests #1657: SUCCESS — full CPU suite, static/docs contracts, Python 3.9/3.10/3.11/3.12 regression matrix, maintained Torch 2.0 CPU regressions, and executable linearmodels==7.0 Stage-B alignment;
  • Maintenance compatibility #670: SUCCESS;
  • Release notes validation #601: SUCCESS;
  • Release package validation #607: SUCCESS, including Ubuntu/Windows/macOS wheel smoke;
  • Benchmark Frontend #674: SUCCESS — Python 3.9/3.11 parser/generator contracts, deterministic staleness, frontend typecheck/build, Chromium e2e, and Chromium/Firefox/WebKit production QA.

Review conclusion

The final applicable-Hausman P2 has been answered with exact physical evidence and resolved. All review threads are resolved.

Under .claude/skills/code-review.md, the final review/fix hard exit is COMPLETE. PR #122 is ready for review/merge from correctness, inference, three-backend behavior, physical acceptance, provenance, artifacts, documentation, and hosted-CI perspectives.

Issue #93 remains open for Stage C covariance expansion/completion.

Merge is intentionally not performed as part of this lifecycle; merging remains a separate explicit action.

@TheHiddenObserver TheHiddenObserver left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fresh post-fix local review checkpoint on exact head 2701aa9feb3796c33c94e6480fcb78c80c6a809c, following .claude/skills/code-review.md changed+adjacent and auto-fix standards. This review does not inherit the earlier merge-ready conclusion.

Review/fix loop findings closed locally:

  • the physical runner now contains a deterministic fitted FE/RE Hausman fixture with a nonzero entity effect that is stably applicable on the NumPy reference, and each requested CUDA backend must produce at least one successful applicable result with finite statistic/p-value/positive df;
  • the dedicated fixture additionally checks FE/RE coefficient and diagnostic-covariance parity plus Hausman statistic/p-value/df parity against NumPy;
  • existing structured-inapplicable Hausman fixtures are retained, so the physical matrix exercises both applicability branches;
  • frontend canonicalization distinguishes the dedicated 48x1 fixture from ordinary balanced/unbalanced Hausman rows and prevents method/case identity collisions;
  • future canonical applicable-Hausman evidence must preserve valid statistic/pvalue/df or the parser fails closed;
  • the new parser contract is explicitly exercised under the Benchmark Frontend Python 3.9/3.11 matrix;
  • physical acceptance/changelog/release-process documentation now treats the older P100/42-row evidence as historical after the validator changed.

Exact-head hosted gates are fully green:

  • Tests #1645: SUCCESS (full CPU, static/docs, Python 3.9-3.12, maintained Torch 2.0 CPU, linearmodels==7.0 alignment);
  • Maintenance compatibility #658: SUCCESS;
  • Release notes validation #589: SUCCESS;
  • Release package validation #595: SUCCESS, including Ubuntu/Windows/macOS wheel smoke;
  • Benchmark Frontend #662: SUCCESS, including Python 3.9/3.11 parser/generator contracts, deterministic staleness, typecheck/build, Chromium e2e, and Chromium/Firefox/WebKit production QA.

Final tree audit from the previous merge-ready head contains only nine expected runner/test/parser/CI/review/changelog/release-process files; temporary review-fix carrier workflows are absent. Review-thread audit leaves exactly one unresolved thread: the applicable-Hausman physical-coverage P2.

Current review result:

  • local CRITICAL: 0
  • local HIGH: 0
  • locally actionable in-scope MEDIUM: 0
  • remote-only blocking finding: 1

The remaining blocker is a fresh exact-clean-head P100 CuPy/Torch execution of the changed dev/benchmarks/validate_panel_stage_b_gpu.py on this SHA. The P2 thread should remain unresolved and the PR should remain Draft until that artifact is audited, canonicalized (44 validation rows), frontend evidence is regenerated, exact-final-head hosted CI is rerun, and a fresh review finds no new issue.

Hard exit: PARTIAL_REMOTE_PENDING. No merge action is taken by this review.

@TheHiddenObserver TheHiddenObserver left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fresh independent final review on exact head cdff57179d97cd53863681cfa46301da919999ec under .claude/skills/code-review.md (auto-fix semantics). This review does not inherit any earlier READY conclusion.

Impact classification / active gates:

  • three-backend behavior: active;
  • inference/correctness: active;
  • benchmark/physical provenance: active;
  • tests/artifacts/docs: active;
  • formula compatibility: verified through existing Stage-B matrix/external alignment where touched;
  • loss/penalty/solver/CV: inactive because this review-fix delta does not alter those components.

Fresh changed+adjacent review result:

  • CRITICAL: 0
  • HIGH: 0
  • relevant MEDIUM: 0
  • unresolved review threads: 0 after closing the applicable-Hausman physical-coverage P2.

Physical correctness/backend gate:

  • exact clean measured implementation: 2701aa9feb3796c33c94e6480fcb78c80c6a809c;
  • raw artifact: results/pr122_p100/panel_stage_b_gpu_validation_2701aa9f.json;
  • artifact commit: 0d0d654d825cea872672f27d02107a58048b345f;
  • raw Git blob: fa3a253e6d882a4e69be29e7e3b1dce7b223b9a9;
  • CuPy and Torch: 17/17 estimator cases and 5/5 Hausman diagnostics each, requested/executed backend identity, no CPU fallback;
  • the four historical Hausman parameterizations remain correctly structured inapplicable/non-PSD;
  • hausman_applicable_nonzero_effect physically executes the applicable statistic/p-value/df path on both backends with df=1 and statistic differences versus NumPy of 7.26e-14 (CuPy) and 1.09e-13 (Torch); FE/RE coefficient/covariance differences are below 1e-12.

Evidence/provenance review:

  • promoted canonical source: results/benchmark_frontend_sources/panel_stage_b_pr122_p100_20260809_2701aa9f.json;
  • SHA-256: 2056f836bfe2a708b3131becca42dbba15e519762c8bafb582000b19f81120bf;
  • source id: panel-stage-b-pr122-20260809-2056f836bfe2;
  • 44 validation-only rows = 17 estimator cases x2 + 5 Hausman diagnostics x2;
  • statistic/pvalue/df for the applicable branch are preserved and fail closed when absent/invalid;
  • no timing or speedup is claimed;
  • previous 42-row source is explicitly superseded historical evidence;
  • frontend and docs generated bundles are byte-identical by Git blob;
  • compare 2701aa9f...cdff5717 contains no statgpu/ numerical source and no physical-runner change, so the P100 artifact remains applicable to the final candidate.

Exact-final hosted gates on cdff5717...:

  • Tests #1657: SUCCESS — full CPU suite, static/docs contracts, Python 3.9/3.10/3.11/3.12 regression matrix, maintained Torch 2.0 CPU regressions, and linearmodels==7.0 Stage-B alignment;
  • Maintenance compatibility #670: SUCCESS;
  • Release notes validation #601: SUCCESS;
  • Release package validation #607: SUCCESS, including Ubuntu/Windows/macOS wheel smoke;
  • Benchmark Frontend #674: SUCCESS — Python 3.9/3.11 generator/parser gates, deterministic staleness, frontend build/typecheck, Chromium e2e, and Chromium/Firefox/WebKit production QA.

Review/fix loop hard exit: COMPLETE. PR #122 is ready for review/merge from correctness, inference, backend parity, physical acceptance, provenance, artifact, documentation, and hosted-CI perspectives. This review does not merge the PR.

@TheHiddenObserver
TheHiddenObserver marked this pull request as ready for review August 9, 2026 14:46
@TheHiddenObserver
TheHiddenObserver merged commit d5dd799 into master Aug 9, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants