refactor: de-duplicate DCC math and cut build_exp5 complexity

[tschm]

Summary

Addresses the three below-10 scorecard findings raised during the rhiza v1.1.3 boost (#470): #471 (code complexity), #472 (architecture / duplication), #473 (baseline test hardening). All in locally-owned book/marimo/notebooks/.

Changes

#472 — de-duplicate the DCC correlation helper (architecture 9→10)

optimize.py's _dcc_correlation reimplemented the exact Engle-DCC covariance/correlation math already defined in Experiment5.py (dcc_correlation → ewm_covariance + correlation_from_covariance). Since optimize.py already executes the notebooks via runpy and those helpers are module-level @app.functions, build_exp5 now pulls dcc_correlation straight from the loaded notebook namespace. The duplicated implementation is deleted — the notebook is the single source of truth.

#471 — cut optimize.py complexity to grade A (complexity 9→10)

• Removing _dcc_correlation eliminates one B(6) block.
• build_exp5's per-day matrix-solve loop is extracted into _solve_positions + _day_position. build_exp5 is now A(1); the helpers are A(4).
• radon cc book/marimo/notebooks/optimize.py reports no block above grade A (was two B(6) blocks). inv_a_norm/solve are still resolved from module globals, so test_build_exp5_skips_singular_days's monkeypatch keeps working.

#473 — harden the pinned-Sharpe builders against dependency drift (test design 9→10)

Our own portfolio assembly used the deprecated polars concat(how="horizontal") (Experiment4.py, Experiment5.py), whose default behaviour is changing in a future release — a silent flip that could perturb the 1e-6-pinned Sharpe baselines. Switched both to the explicit, stable how="horizontal_extend" (matching optimize.py) and documented the drift-resistance rationale in SHARPE_PINS.md. Row heights are equal, so the result is numerically identical. (The tolerance itself is already centralized in expected_sharpe.py and documented in SHARPE_PINS.md, which explicitly forbids widening it — so no tolerance change was made.)

Verification

All gates pass:

Gate	Result
make fmt	✅ PASS
make typecheck	✅ PASS
make docs-coverage	✅ PASS (100%)
make deptry	✅ PASS
make security	✅ PASS
make test	✅ PASS (76 passed, 100% coverage)

The pinned Sharpe baselines (both notebook-end-to-end and the build_exp* builders, all 5 experiments) are unchanged, confirming the refactor and the concat switch are behaviour-preserving. The 120 remaining DeprecationWarnings now originate solely from jquantstats internals (upstream); our code contributes none.

Closes #471
Closes #472
Closes #473

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

@tschm, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fe34780d-f342-47f1-b1ec-b1556b62236b

📥 Commits

Reviewing files that changed from the base of the PR and between 5b833d6 and c1f1846.

📒 Files selected for processing (4)

• book/marimo/notebooks/Experiment4.py
• book/marimo/notebooks/Experiment5.py
• book/marimo/notebooks/optimize.py
• docs/development/SHARPE_PINS.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/optimize-complexity-dedup

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

This PR refactors the Optuna-based optimizer for the marimo CTA notebooks to remove duplicated Engle-DCC math and reduce cyclomatic complexity, while also hardening the pinned-Sharpe portfolio builders against Polars API default drift.

Changes:

• Removed the duplicated DCC correlation implementation from optimize.py and re-used Experiment5.py’s dcc_correlation via the executed notebook namespace.
• Reduced build_exp5 complexity by extracting per-day position solving into _solve_positions and _day_position, preserving the “skip degenerate days” behavior.
• Switched notebook portfolio assembly to pl.concat(..., how="horizontal_extend") (and documented the rationale) to avoid future default changes affecting pinned Sharpe baselines.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
docs/development/SHARPE_PINS.md	Documents why builders should prefer explicit, stable dependency APIs (Polars concat mode) to protect pinned-Sharpe baselines.
book/marimo/notebooks/optimize.py	De-duplicates DCC math by reusing Experiment 5 helpers and reduces build_exp5 complexity via extracted helpers.
book/marimo/notebooks/Experiment5.py	Updates portfolio assembly concat mode to horizontal_extend for stability.
book/marimo/notebooks/Experiment4.py	Updates portfolio assembly concat mode to horizontal_extend for stability.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.