|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Commands |
| 6 | + |
| 7 | +A `.venv` holds the scientific core plus the optional stack; the core alone |
| 8 | +(numpy, pandas, scipy, scikit-learn, matplotlib, pyarrow) is enough for tests |
| 9 | +and lint. Run everything from the repo root. |
| 10 | + |
| 11 | +- Tests: `.venv/bin/python -m pytest tests/ -q` (pytest config sets |
| 12 | + `pythonpath = ["."]`, so the top-level packages resolve from the root). |
| 13 | +- Single test: `.venv/bin/python -m pytest tests/test_weight_interface.py::test_equal_weight_sums_to_one -v` |
| 14 | +- Lint: `.venv/bin/ruff check .` (CI enforces this; must stay clean). Auto-fix: `ruff check . --fix`. |
| 15 | +- Operational scripts import the top-level packages directly, so run them with |
| 16 | + the root on the path: `PYTHONPATH=. .venv/bin/python scripts/<name>.py` |
| 17 | + (validate_performance, generate_report, survivorship_demo, verify_brokers, |
| 18 | + paper_trade_cycle). `generate_report.py` and `validate_performance.py --pit` |
| 19 | + print results; `--live` refetches data instead of using the bundled snapshot. |
| 20 | +- CI (`.github/workflows/ci.yml`) runs ruff + pytest on Python 3.11/3.12 with |
| 21 | + core deps only; optional extras are deliberately not installed there. |
| 22 | + |
| 23 | +## The weight contract (keystone: portfolio/base.py) |
| 24 | + |
| 25 | +Every component that emits weights validates them through one of two functions; |
| 26 | +which one depends on where it sits in the pipeline: |
| 27 | + |
| 28 | +- `enforce_weight_contract` (STRICT): float64 `(n_assets,)`, each in `[-1, 1]`, |
| 29 | + sum `== 1.0` long-only or `== 0.0` market-neutral. This is the **allocation** |
| 30 | + layer contract; `PortfolioOptimizer.optimize` applies it automatically, so |
| 31 | + subclasses only implement `_compute_weights` returning a raw vector that |
| 32 | + already satisfies the contract for the configured mode. |
| 33 | +- `enforce_exposure_contract` (RELAXED): same box, but gross (`sum |w|`) `<= cap` |
| 34 | + and the sum may be below 1 (the remainder is cash). This is the contract for |
| 35 | + **timing/risk overlays and the post-overlay strategy output**, because |
| 36 | + overlays legitimately scale exposure down (a flat book sums to 0, a |
| 37 | + half-scaled long-only book to 0.5). |
| 38 | + |
| 39 | +Getting this distinction wrong is the most common mistake here: a regime-gated |
| 40 | +or vol-scaled book is NOT required to sum to 1. Violations raise |
| 41 | +`WeightContractViolationError` immediately. |
| 42 | + |
| 43 | +## The pipeline (strategies/base_strategy.py) |
| 44 | + |
| 45 | +`w_t = R_t( T_t( A_t( S_t( X<=t ) ) ) )`: Select -> Allocate -> Timing -> Risk. |
| 46 | +- Subclass `Strategy` and implement `select(ctx) -> pd.Series` (alpha scores |
| 47 | + indexed by the chosen symbols). Override `allocate` for score-driven weights; |
| 48 | + the default delegates to the injected `PortfolioOptimizer`. |
| 49 | +- Timing and risk overlays are `callable(weights, ctx) -> weights`; concrete |
| 50 | + strategies wire components (HMMRegime, VIXScaler, CircuitBreaker, ...) with |
| 51 | + small lambdas that pull the right data out of the `StrategyContext`. |
| 52 | +- `generate_weights(prices, rebalance_dates)` produces the date-by-symbol target |
| 53 | + panel that the backtest engines consume. |
| 54 | + |
| 55 | +## Conventions that are load-bearing (violating them breaks the build) |
| 56 | + |
| 57 | +- **Lazy imports.** Every heavy/optional dependency (torch, lightgbm/xgboost/ |
| 58 | + catboost, hmmlearn, transformers, stable-baselines3, gymnasium, alpaca-trade- |
| 59 | + api, ib_insync, ccxt, yfinance, fredapi, polygon, redis, sqlalchemy, lxml) is |
| 60 | + imported *inside* the method that uses it, with an offline fallback or a clear |
| 61 | + ImportError. Modules must import with only the scientific core. Do not add a |
| 62 | + heavy import at module top level. |
| 63 | +- **Mandatory costs.** Backtest engines (`backtest/engines/`) raise if |
| 64 | + constructed without a `TransactionCostModel`. Slippage in that model is a flat |
| 65 | + per-trade rate; size/impact-aware costs live in `execution_models/market_impact.py`. |
| 66 | +- **Strict causality / PIT.** Factors and features use only past data; |
| 67 | + `pit_enforcer.py` keys fundamentals off `announcement_date`, |
| 68 | + `lookahead_detector.py` scans for leakage, and `walk_forward.py` applies a |
| 69 | + purge + embargo gap. |
| 70 | +- **Determinism.** `timing/hmm_regime.py` pins BLAS to one thread (threadpoolctl) |
| 71 | + around the HMM fit so backtests are bit-for-bit reproducible; a non-converged |
| 72 | + EM near a regime boundary otherwise flips under multithreaded float ordering. |
| 73 | + Keep model fits seeded and deterministic. |
| 74 | +- **Reproducible results.** `generate_report.py` and the README "Results" read |
| 75 | + the fixed snapshot `data/sample/rotation_prices.csv`, because live yfinance |
| 76 | + re-adjusts historical closes on every fetch. Quote numbers from the generator |
| 77 | + verbatim rather than hand-typing them. |
| 78 | + |
| 79 | +## Layer ABCs to subclass |
| 80 | + |
| 81 | +`DataProvider` (data/providers/base.py): `fetch_ohlcv/fetch_fundamentals/ |
| 82 | +fetch_macro`, canonical UTC-naive OHLCV schema. `Universe` (data/universe/ |
| 83 | +base.py): point-in-time membership; `SP500Universe.from_wikipedia()` |
| 84 | +reconstructs real historical constituents (survivorship-safe membership). |
| 85 | +`Broker` (execution/brokers/base.py): `submit_order/get_positions/get_account`, |
| 86 | +adapters lazy-load their SDK. `OrderManager`: a NEW -> SUBMITTED -> FILLED state |
| 87 | +machine that validates the transition before mutating the order. |
| 88 | + |
| 89 | +## Known structural issue |
| 90 | + |
| 91 | +The eight packages are TOP-LEVEL (`from portfolio.base import ...`, |
| 92 | +`import data`), not namespaced under a `quantcortex/` package. This works via |
| 93 | +`pythonpath = ["."]` but squats generic names and is hostile to `pip install`. |
| 94 | +Do not paper over it by reordering imports; the proper fix is a deliberate, |
| 95 | +repo-wide move into a `quantcortex/` package (its own reviewed change). |
| 96 | + |
| 97 | +## Honesty norms |
| 98 | + |
| 99 | +The strategies' Sharpe targets (1.10 / 0.9 in the README) are aspirational |
| 100 | +design goals; the measured baselines (rotation ~0.17, momentum_ml ~0.63) are |
| 101 | +reported honestly and are NOT to be tuned toward a single backtest (that is the |
| 102 | +overfitting the Deflated Sharpe Ratio and BHY tooling exist to catch). See |
| 103 | +PERFORMANCE.md. Source and docs are ASCII-only (no em-dashes, en-dashes, or |
| 104 | +arrows); the README directory-tree block is the one exception (box-drawing). |
0 commit comments