Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,6 @@ htmlcov/
.DS_Store
.idea/
.vscode/

# Regenerated walk-forward example CSVs
/walk-forward-output/
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,22 @@ Seed 42 happens to be one of the 14. With no trends in the data, the crossover m

![Equity curves and drawdowns on synthetic data](docs/images/backtest_equity.png)

**Walk-forward portfolio construction**
([`walk_forward_risk_parity.py`](examples/walk_forward_risk_parity.py)). Connect
trailing-return estimation, risk-parity allocation and backtesting on fictional
prices. Each monthly allocation uses only closes preceding the rebalance date.
A monthly equal-weight benchmark shares the same cash warmup, dates and costs.

```bash
uv run python examples/walk_forward_risk_parity.py
```

The command exports prices, both target tables, estimation windows, net wealth
and a common-period summary to `walk-forward-output/`. The
[walkthrough](docs/WALK_FORWARD.md) explains the timing, CSV fields, reproducible
inputs and limitations. These exports demonstrate mechanics, not investment
performance.

## How it works

```mermaid
Expand Down
133 changes: 133 additions & 0 deletions docs/WALK_FORWARD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Walk-forward portfolio construction

This runnable example connects quantkit's existing covariance estimator,
risk-parity optimizer and backtest engine. It builds a new allocation each
month using a trailing history, then evaluates that allocation on subsequent
returns. All prices describe fictional assets; no market data is downloaded.

After `uv sync`, run from the repository root:

```bash
uv run python examples/walk_forward_risk_parity.py
```

The default run writes six CSVs to `walk-forward-output/`. To choose a location
or run a shorter example:

```bash
uv run python examples/walk_forward_risk_parity.py --days 400 --lookback 60 --seed 7 --cost-bps 10 --output-dir /tmp/quantkit-walk-forward
```

The CLI prints the simulation inputs and written paths. It exits 1 on an
invalid window, insufficient history, invalid costs or an I/O error. No plotting
extra, database, credentials or network connection is needed after installation.

## The scenario

Three fictional assets, `AAA`, `BBB` and `CCC`, have different simulated
volatilities. The workflow asks how to turn an estimated covariance matrix into
a sequence of portfolio allocations, while keeping the estimation and
subsequent evaluation periods separate.

| Input | Default |
| --- | --- |
| Prices | 1,260 business-day closes from 2020-01-01, seed 42 |
| Annual GBM drift | 4%, 6%, 8% for AAA, BBB, CCC |
| Annual GBM volatility | 10%, 18%, 28% |
| Pairwise Brownian correlation | 0.3 |
| Estimation | 252 prior simple returns, Ledoit–Wolf covariance |
| Risk parity | Long-only weights with equal estimated variance contributions |
| Benchmark | Equal capital weights, rebalanced on the same dates |
| Rebalancing | First business-day close of each month with sufficient history |
| Starting wealth | 1 in each portfolio |
| Trading costs | 10 basis points of traded notional, including initial entry |
| Idle cash | Zero interest |

These are simulation settings, not asset forecasts or recommended allocations.
Equal estimated risk contribution does not mean equal capital weights or equal
realized risk contributions in the following month.

## The information boundary

For a rebalance at price-table position **t**:

1. Read `lookback + 1` closes ending at **t−1**. They produce exactly
`lookback` simple returns; neither close t nor later data enters the
covariance estimate.
2. Estimate the covariance with `ledoit_wolf_covariance` and construct targets
with `risk_parity`. Set equal-weight targets on that same date.
3. The existing `backtest_weights` engine trades at close **t**, charges costs
there, and holds those weights during **t+1**. Between trades, share counts
stay fixed and portfolio weights drift.

The one-close estimation lag is deliberate: the target can be prepared before
the rebalance close. The fill still uses that close's price under the engine's
existing convention. There is no attempt to model an actual closing auction.

Both portfolios start in cash and wait for the **same** first eligible monthly
rebalance. They do not invest immediately when the rolling window fills partway
through a month. In the default run, the first trade is **2021-01-01**; its 252
returns come from closes **2020-01-14 through 2020-12-31**. Exposure begins on the
following synthetic business day. Each portfolio pays the same 0.1% initial
entry cost, reducing initial wealth from 1 to 0.999 at the first trade.

## Inspect the CSVs

| File | Contents |
| --- | --- |
| `prices.csv` | The exact fictional daily closes used by both portfolios |
| `risk_parity_targets.csv` | Date-indexed target weight per asset; blank rows mean no trade |
| `equal_weight_targets.csv` | Benchmark targets with the identical rebalance calendar |
| `estimation_windows.csv` | Each rebalance date, first/last historical close, and return count |
| `wealth.csv` | Both net wealth curves on the full date index, including cash warmup |
| `summary.csv` | Existing performance metrics for the shared evaluation period |

Targets are decisions made at a close, **not** weights held during that same
day. Blank rows keep existing shares unchanged; during the initial warmup they
leave the portfolio in cash. A zero target would instead instruct the engine
to sell a position. The target tables include all dates so this distinction is
inspectable.

The summary starts at the first trade close and ends at the final close. It
includes initial entry costs and excludes the earlier all-cash warmup from
annualization. Both strategies use exactly that period, 252 periods per year
and a zero risk-free rate. The full wealth exports retain the warmup; their
final values minus 1 match the summary's total returns. Ratio metrics can be
blank when their denominators are zero, following the library's NaN convention.

Inspecting these metrics demonstrates accounting and timing. A difference
between the curves on one synthetic history is not evidence of investment
performance, an alpha claim or a recommendation to trade.

## Assumptions and limits

The price simulator uses correlated geometric Brownian motion with constant
drift and volatility. Its business dates omit weekends but include exchange
holidays. It has no fat tails, volatility clustering, changing asset universes
or missing prices. The estimated covariance can still differ from subsequent
realized covariance; shrinkage and risk parity do not remove estimation risk.

The engine uses daily close-to-close returns, same-close fills and proportional
costs. It does not model slippage, market impact, auction fills, taxes or cash
interest. The example makes no change to those documented
[engine assumptions](DESIGN.md). Results describe only the specified fictional
sample and conventions.

The example preserves the existing public APIs. Its tests perturb current and
future prices to check that prior targets stay unchanged, check the shared
cash warmup and feasible weights, and verify that adding costs cannot improve
wealth when targets are held fixed. Actual subprocess CLI runs verify the
export schema and deterministic files:

```bash
uv run pytest tests/test_walk_forward_example.py -q
```

Regeneration is deterministic with a fixed seed and the dependencies in
`uv.lock`. To reproduce the default exports in two directories:

```bash
uv run python examples/walk_forward_risk_parity.py --output-dir /tmp/quantkit-first
uv run python examples/walk_forward_risk_parity.py --output-dir /tmp/quantkit-second
diff -r /tmp/quantkit-first /tmp/quantkit-second
```
40 changes: 40 additions & 0 deletions docs/superpowers/plans/2026-10-25-walk-forward-risk-parity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Walk-Forward Risk-Parity Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make a prior-information-only portfolio construction/backtest workflow runnable and exportable.

**Architecture:** An additive example composes `compute_returns`, `ledoit_wolf_covariance`, `risk_parity`, `rebalance_days`, `backtest_weights` and `performance_summary`. Preserve the library APIs.

**Tech Stack:** Python 3.11+, NumPy/pandas and existing quantkit APIs; no new dependencies.

**Spec:** `docs/superpowers/specs/2026-10-25-walk-forward-risk-parity-design.md`

## Global Constraints

- Defaults: 1,260 fictional business days, 252-return window, seed 42, 10 bps costs, initial wealth 1.
- At date t use only closes ending t−1; trade at close t and hold during t+1.
- Shared monthly rebalance calendar, cash warmup and costs for risk parity/equal weight.
- Export prices, both target tables, wealth, summary and estimation-window CSVs.
- No optimization rewrite, live data, performance claim or execution capability.

## Review Focus

- Using date t's close in covariance must fail the current-close perturbation test.
- Insufficient history must produce a clear CLI failure before files are written.
- Both strategies must have identical trade dates and warmup treatment.
- Higher costs with unchanged targets must never improve wealth.
- Exported target blanks mean no trade, and summary excludes the shared cash warmup.

### Task 1: Example and scenario

**Files:** Create `examples/walk_forward_risk_parity.py`, `tests/test_walk_forward_example.py`, `docs/WALK_FORWARD.md`; modify `README.md` and `.gitignore` for the default output directory.

**Interfaces:** `build_targets(prices: pd.DataFrame, lookback: int) -> TargetSchedule` with risk-parity/equal-weight frames and estimation windows; `run_example(output_dir: Path, *, n_days: int, lookback: int, cost_bps: float, seed: int) -> dict[str, Path]`; `main(argv: Sequence[str] | None = None) -> int`.

- [ ] Write meaningful failing tests for prior-information invariance, feasible allocations, shared cash warmup, costs, windows and deterministic actual CLI files.
- [ ] Run `uv run pytest tests/test_walk_forward_example.py -q`; expect failure on the absent example.
- [ ] Implement the minimal typed composition and export workflow; rerun focused tests.
- [ ] Add README scenario, output contract, explicit timing and limits with the measured run dates.
- [ ] Run the full `uv run pytest --cov=quantkit --cov-fail-under=90`, existing ruff/format/mypy checks and the documented workflow. Expect all existing 333 tests plus new tests to pass.
- [ ] Fetch and preserve concurrent main additions, commit, push a scoped feature PR and attach it; parent reviews the exact head, no merge.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Walk-forward risk-parity example

Add a runnable educational example joining existing portfolio estimation and backtesting APIs. Generate deterministic fictional AAA/BBB/CCC prices, estimate a trailing covariance at each monthly rebalance, construct risk-parity targets, and compare a monthly equal-weight benchmark on identical dates, initial capital, warmup and proportional costs. No statistical or optimization primitive changes.

At the close of date t, the estimator uses exactly `lookback` simple returns from `lookback + 1` closes ending at t−1. The engine trades at close t and holds those targets during t+1. The extra one-close lag makes the information boundary explicit. Both strategies remain in zero-interest cash until the first monthly rebalance with sufficient history, then drift between rebalances. Long-only targets sum to one.

Export fictional prices, both daily target tables (blank rows mean no trade), daily wealth, evaluation-period summary and estimation-window dates as CSVs. Summaries start on the shared first trade date, including its entry cost; full wealth includes the shared cash warmup. Defaults: 1,260 business days, 252-return window, seed 42, 10 bps costs, initial wealth 1. No plots or extra dependencies.

Approaches considered: a new strategy class would expand the library API; a notebook adds tooling; a standalone example is selected to keep composition and its assumptions inspectable. Tests cover future/current-close shocks, feasible allocations, shared cash warmup, fixed-target cost monotonicity, auditable estimation windows and deterministic actual CLI exports. Document GBM assumptions, synthetic data, same-close fills, covariance-estimation risk and the absence of any performance or investment claim.
176 changes: 176 additions & 0 deletions examples/walk_forward_risk_parity.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
"""Walk-forward risk parity against equal weight on fictional prices.

Run ``uv run python examples/walk_forward_risk_parity.py``. Each monthly target
uses only closes preceding the rebalance date; the existing engine trades at
that date's close and holds the target during the following day. See
``docs/WALK_FORWARD.md`` for the timing, CSV contract and model limitations.
"""

from __future__ import annotations

import argparse
import sys
from collections.abc import Sequence
from dataclasses import dataclass
from pathlib import Path

import numpy as np
import pandas as pd

from quantkit import QuantkitError
from quantkit.backtest import backtest_weights, performance_summary, rebalance_days
from quantkit.data import simulate_gbm_prices
from quantkit.portfolio import compute_returns, ledoit_wolf_covariance, risk_parity


@dataclass(frozen=True)
class TargetSchedule:
"""Daily targets (NaN means no trade) and the estimation windows behind them."""

risk_parity: pd.DataFrame
equal_weight: pd.DataFrame
windows: pd.DataFrame


def build_targets(prices: pd.DataFrame, lookback: int = 252) -> TargetSchedule:
"""Estimate monthly targets from exactly ``lookback`` returns ending at t-1.

A return window needs one more closing price than returns. At rebalance
position t the price window is [t-lookback-1, t), excluding close t. Both
strategies remain in cash until the first monthly rebalance with that history.
"""
if lookback < 2:
raise ValueError("lookback must be at least 2 returns")
if not isinstance(prices.index, pd.DatetimeIndex):
raise ValueError("prices need a DatetimeIndex")
# compute_returns validates the whole input. Its first row corresponds to
# original price position 1, hence the additional -1 in the slice below.
returns = compute_returns(prices)
risk_targets = pd.DataFrame(np.nan, index=prices.index, columns=prices.columns)
equal_targets = risk_targets.copy()
windows: list[dict[str, object]] = []
for offset in np.flatnonzero(rebalance_days(prices.index, "M")):
position = int(offset)
if position < lookback + 1:
continue
history = returns.iloc[position - lookback - 1 : position - 1]
covariance = ledoit_wolf_covariance(history)
risk_targets.iloc[position] = risk_parity(covariance).reindex(prices.columns)
equal_targets.iloc[position] = 1.0 / len(prices.columns)
windows.append(
{
"rebalance_date": prices.index[position],
"history_start": prices.index[position - lookback - 1],
"history_end": prices.index[position - 1],
"n_returns": len(history),
}
)
return TargetSchedule(
risk_targets,
equal_targets,
pd.DataFrame(
windows,
columns=["rebalance_date", "history_start", "history_end", "n_returns"],
),
)


def run_example(
output_dir: Path,
*,
n_days: int = 1260,
lookback: int = 252,
cost_bps: float = 10.0,
seed: int = 42,
) -> dict[str, Path]:
"""Simulate, estimate, backtest and export six inspectable CSV files."""
prices = simulate_gbm_prices(
tickers=["AAA", "BBB", "CCC"],
n_days=n_days,
drift=[0.04, 0.06, 0.08],
vol=[0.10, 0.18, 0.28],
correlation=0.3,
seed=seed,
)
schedule = build_targets(prices, lookback)
trade_positions = np.flatnonzero(schedule.risk_parity.notna().all(axis=1).to_numpy())
if len(trade_positions) == 0 or int(trade_positions[0]) >= len(prices) - 1:
raise ValueError("not enough history for a monthly rebalance and subsequent return")
first_trade = int(trade_positions[0])
targets = {
"risk_parity": schedule.risk_parity,
"equal_weight": schedule.equal_weight,
}
results = {
name: backtest_weights(prices, weights, cost_bps=cost_bps)
for name, weights in targets.items()
}
# Start both summaries at the first trade, including its entry cost. The
# exported wealth still shows every day, including the shared cash warmup.
summary = pd.DataFrame(
{
name: performance_summary(
result.returns.iloc[first_trade:],
turnover=result.turnover.iloc[first_trade:],
)
for name, result in results.items()
}
)
summary.index.name = "metric"
wealth = pd.DataFrame({name: result.equity for name, result in results.items()})
tables = {
"prices.csv": prices,
"risk_parity_targets.csv": schedule.risk_parity,
"equal_weight_targets.csv": schedule.equal_weight,
"wealth.csv": wealth,
"summary.csv": summary,
"estimation_windows.csv": schedule.windows,
}
output_dir.mkdir(parents=True, exist_ok=True)
written = {}
for name, table in tables.items():
path = output_dir / name
table.to_csv(
path,
index=name != "estimation_windows.csv",
date_format="%Y-%m-%d",
float_format="%.12g",
lineterminator="\n",
)
written[name] = path
return written


def main(argv: Sequence[str] | None = None) -> int:
"""Run the fictional workflow; invalid inputs or I/O errors exit 1."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output-dir", type=Path, default=Path("walk-forward-output"))
parser.add_argument("--days", type=int, default=1260)
parser.add_argument("--lookback", type=int, default=252, help="number of prior daily returns")
parser.add_argument("--cost-bps", type=float, default=10.0)
parser.add_argument("--seed", type=int, default=42)
args = parser.parse_args(argv)
try:
paths = run_example(
args.output_dir,
n_days=args.days,
lookback=args.lookback,
cost_bps=args.cost_bps,
seed=args.seed,
)
except (QuantkitError, ValueError, OSError) as exc:
print(f"walk-forward: error: {exc}", file=sys.stderr)
return 1
print(
f"SYNTHETIC data: {args.days} business-day closes, seed {args.seed}; "
f"{args.lookback} prior returns, monthly rebalancing, costs {args.cost_bps:g} bps."
)
print("Both portfolios share the cash warmup and evaluation dates.")
print("These exports demonstrate mechanics, not evidence of investment performance.")
for path in paths.values():
print(f"wrote {path}")
return 0


if __name__ == "__main__":
raise SystemExit(main())
Loading
Loading