Add a strategy optimizer for the cash-deploy backtest - #3
Merged
Conversation
Bayesian (Optuna TPE) search over the cash-deploy reserve, refill rate and drawdown deploy tranches to find configs that beat the index, with walk-forward validation for an honest out-of-sample estimate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
study.best_value raises until a trial completes; skip the progress report in that window so a pruning objective can never crash the search. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an Optuna-driven optimizer and walk-forward validation workflow for the cash-deploy backtest, plus a new Streamlit “Optimizer” page, so strategy parameters can be searched automatically and reported with out-of-sample results.
Changes:
- Introduces a new engine module
optimizer.pyimplementing Optuna search, multiple objective functions, and walk-forward validation. - Adds a new Streamlit page to run the optimizer, display recommended parameters, fold results, and charts.
- Refactors Sharpe-vs-cash into
metrics.sharpe_vs_cash, adds Optuna dependency, and expands package-root exports.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Locks Optuna and its transitive dependencies (SQLAlchemy/Alembic/etc.). |
| pyproject.toml | Adds optuna>=3.6 to project dependencies. |
| src/portfolio_research_lab/optimizer.py | New optimizer engine (search space encoding, objectives, optimize + walk-forward). |
| src/portfolio_research_lab/metrics.py | Adds reusable sharpe_vs_cash() helper. |
| src/portfolio_research_lab/init.py | Re-exports optimizer + cash-deploy symbols from package root. |
| app/pages/2_Optimize.py | New Streamlit page to run optimization and render results/visuals. |
| app/pages/1_Cash_Deploy.py | Replaces local Sharpe-vs-cash helper with metrics.sharpe_vs_cash(). |
| tests/test_optimize.py | New test suite covering encoding invariants, objective behavior, optimize(), walk_forward(). |
| tests/test_metrics.py | Adds unit tests for metrics.sharpe_vs_cash(). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- optimize() raises when split<1.0 leaves no holdout instead of silently dropping the test slice - compute run metrics once per trial (objective takes the metrics dict) - move Optuna verbosity change out of import into optimize() - pass on_trial through to walk_forward so the page progress bar updates Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
optimizer.py: an Optuna (TPE Bayesian) search with a threshold-grid encoding that makes every sampledDeployRulevalid by construction, a selectable objective (excess CAGR vs index by default, plus a drawdown-capped variant, Sharpe-vs-cash, and raw CAGR), and the optimizer choosing the number of buckets (1–5).app/pages/2_Optimize.pypage: button-gated search with progress, the recommended config + bucket table, per-fold train-vs-test results, comparison against 100% stocks and the four presets, equity + search-history charts, and prominent curve-fitting / price-return-only warnings._sharpe_vs_cashfrom the cash-deploy page intometrics.sharpe_vs_cash; added Optuna as a dependency; re-exported the optimizer and cash-deploy engine from the package root.How to test
In the app, open the Optimizer page, pick a start year and a small trials/folds budget, press Run search, and confirm the recommended config, walk-forward table, comparison table, and charts render, and that the out-of-sample column is shown.
Security review
No security-impacting changes.