diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..0478ab8 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,26 @@ +# Changelog + +This file documents relevant project changes for public releases. + +## [0.1.0] - YYYY-MM-DD + +### Added + +- Packaging structure under `src/`. +- Consolidated public API for variance testing, normalization, battery execution, simulation, and visuals. +- Support for `input_kind="log_prices"` and `input_kind="returns"` in the public variance-ratio workflow. +- Common input normalization utilities shared by test orchestration paths. +- Weak-form battery v1 with variance ratio multi-q, Holm family summary, Ljung-Box variants, runs test on signs, and ARCH LM. +- Offline examples under `examples/`. +- Minimal CI coverage for core quality checks. + +### Changed + +- Variance ratio implementation was hardened for robustness across edge cases. +- Non-computable scenarios are handled explicitly with structured outcomes and warnings instead of silent failures. +- Rolling support was introduced partially for variance ratio multi-q, Ljung-Box returns, and Ljung-Box squared returns. + +### Notes + +- Rolling is intentionally not applied in this version to Holm summary, runs test, or ARCH LM. +- Release `0.1.0` is documented as ready for maintainer-driven publication workflows. diff --git a/README.md b/README.md index e580e7f..766e9cd 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,58 @@ -# Variance Ratio Test [![ForTheBadge built-with-science](http://ForTheBadge.com/images/badges/built-with-science.svg)](https://GitHub.com/Naereen/) +# variance-test -[![Python Version](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://shields.io/) [![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity) [![Documentation Status](https://readthedocs.org/projects/ansicolortags/badge/?version=latest)](http://ansicolortags.readthedocs.io/?badge=latest) +## Short description -A comprehensive implementation of the Variance Ratio Test following Lo & MacKinlay (1988), providing statistical tools for examining the stochastic evolution of financial log price series. The test evaluates the Random Walk Hypothesis (Efficient Markets Hypothesis) by comparing variance estimators at different sampling intervals. +`variance-test` provides a focused toolkit for weak-form market efficiency diagnostics. The current package includes a variance ratio test, common series normalization utilities, a weak-form battery, rolling-window support for a subset of battery tests, and offline simulation utilities with runnable examples. -## Overview +## Installation -The Variance Ratio Test investigates market efficiency by testing whether log price increments follow a random walk. The implementation supports testing under different null hypotheses to assess the quality of market efficiency: +This repository is documented for the `0.1.0` release target. -### Null Hypotheses +Install from PyPI (target command for the public package): -- **Homoskedastic Increments** (*strong market efficiency*): The disturbances/increments are IID (independently and identically distributed) normal random variables, where the variance of increments is a linear function of the observation interval. This hypothesis corresponds to the Brownian Motion model. +```bash +pip install variance-test +``` -![homocedasticity](img/Homocedasticity.png) +Install locally in editable mode: -- **Heteroskedastic Increments** (*semi-strong market efficiency*): The disturbances/increments are independent but not identically distributed (INID). The variance of increments is a non-linear function of the observation interval. This hypothesis corresponds to the Heston Model. +```bash +pip install -e . +``` -![heterocedasticty](img/Heteroscedasticity.png) +## Current scope -- **Model-Dependent Increments** (*weak market efficiency*): The third form relaxes the independence assumption, allowing for conditional heteroskedastic increments. The volatility has either a non-linear structure (conditional on itself) or is conditional on another random variable. Stochastic processes employing ARCH (Autoregressive Conditional Heteroscedasticity) and GARCH (Generalized AutoRegressive Conditional Heteroscedasticity) models belong to this category. +The current public scope includes: -![rw3](img/rw3.png) +- `EMH().vrt(...)` for variance ratio testing. +- `normalize_series(...)` for shared input normalization. +- `run_weak_form_battery(...)` for the weak-form battery v1. +- Rolling support for: + - variance ratio multi-q + - Ljung-Box returns + - Ljung-Box squared returns +- Offline examples under `examples/`. -## Installation +## Weak-form battery -### Requirements +The weak-form battery v1 currently includes: -```bash -pip install -e . -``` +- variance ratio multi-q +- Holm summary for the VR family +- Ljung-Box returns +- Ljung-Box squared returns +- runs test on signs +- ARCH LM -### Python Version +Rolling support in this version does not apply to: -Python 3.11 or higher is required. +- Holm summary +- runs test +- ARCH LM ## Usage -### Basic Usage with EMH Class +### Variance ratio with `EMH().vrt(...)` ```python import numpy as np @@ -44,32 +60,16 @@ from variance_test import EMH emh = EMH() -# Modo log-prices (default histórico) -log_prices = np.log(np.cumsum(np.random.randn(201)) * 0.01 + 100) -z_score, p_value = emh.vrt(X=log_prices, q=4, input_kind="log_prices") - -# Modo returns -returns = np.diff(log_prices) -z_score, p_value = emh.vrt(X=returns, q=4, input_kind="returns") +log_prices = np.log(np.cumsum(np.random.normal(0.0, 1.0, 600)) + 5000.0) +z_score, p_value = emh.vrt( + X=log_prices, + q=4, + input_kind="log_prices", +) +print(z_score, p_value) ``` -`EMH.vrt` interpreta `X` como **log-prices** por default para mantener compatibilidad histórica. -Desde este sprint también acepta **returns** con `input_kind="returns"`; en ese caso, -reconstruye internamente una serie sintética de log-prices con origen `0.0` y aplica -el mismo VRT. - - -### Weak-form Battery v1 - -The package now includes an executable weak-form efficiency battery via `run_weak_form_battery(...)`. - -Battery v1 includes: -- variance ratio multi-q -- Ljung-Box returns -- Ljung-Box squared returns -- runs test on signs -- ARCH LM -- Holm summary for the VR family +### Weak-form battery with `run_weak_form_battery(...)` ```python import numpy as np @@ -78,24 +78,16 @@ from variance_test import BatteryConfig, run_weak_form_battery rng = np.random.default_rng(42) returns = rng.normal(0.0, 0.01, size=2000) -config = BatteryConfig(input_kind="returns", q_list=(2, 4, 8), ljung_box_lags=(5, 10, 20)) +config = BatteryConfig( + input_kind="returns", + q_list=(2, 4, 8), + ljung_box_lags=(5, 10, 20), +) outcome = run_weak_form_battery(returns, config=config) print(outcome.multiple_testing["battery_summary"]) ``` -> Warning: this battery provides evidence against compatibility with weak-form efficiency **under this battery**. -> It is **not** definitive proof of market inefficiency. -> -Rolling mode is available by setting `BatteryConfig(rolling_window=..., rolling_step=...)`. -In this version, rolling results are provided only for: -- variance ratio multi-q -- Ljung-Box returns -- Ljung-Box squared returns - -Rolling mode is **not** implemented in this version for: -- Holm summary -- runs test -- ARCH LM +### Weak-form battery with rolling ```python import numpy as np @@ -115,225 +107,35 @@ outcome = run_weak_form_battery(returns, config=config) print(outcome.rolling["n_windows"]) ``` -Offline runnable examples are available under `examples/`: +## Examples + +Available offline scripts: + - `examples/basic_battery.py` - `examples/rolling_battery.py` - `examples/variance_ratio_only.py` -- `examples/empirical_application.py` - -### Running Simulations - -The package includes a comprehensive simulation framework to test the VRT on different stochastic processes: - -```bash -python -m variance_test.simulation --series 10 --horizon 100 --aggregation 3 --seed 42 --no-plot -``` - -#### Command Line Arguments - -- `--series`: Number of trajectories to simulate per model (default: 10) -- `--horizon`: Number of observations per trajectory (default: 100) -- `--initial-price`: Initial price for each trajectory (default: 100.0) -- `--mu`: Mean of return trajectories (default: 0.09) -- `--sigma`: Base volatility (default: 0.1) -- `--jump-intensity`: Jump intensity for Merton process (default: 50.0) -- `--kappa`: Mean reversion speed for Heston model (default: 0.1) -- `--theta`: Target variance level in Heston (default: 0.06) -- `--rf`: Risk-free rate for Heston (default: 0.02) -- `--aggregation`: Aggregation horizon for VRT statistic (default: 5) -- `--homoskedastic`: Use homoskedastic null hypothesis instead of heteroskedastic -- `--seed`: Random seed for reproducibility -- `--no-plot`: Skip displaying density comparison plot - -### Simulation Example with Python - -```python -from variance_test import SimulationConfig, run_simulation - -config = SimulationConfig( - num_series=5, - horizon=200, - initial_price=100.0, - mu=0.05, - sigma=0.2, - aggregation_horizon=2, - heteroskedastic=False, - seed=42 -) - -results = run_simulation(config) - -print(f"Generated {len(results.processes)} price processes") -print(f"Mean z-score: {results.z_scores.mean():.4f}") -print(f"Mean p-value: {results.p_values.mean():.4f}") -``` - -## Implementation Details - -### Variance Ratio Test - -The implementation follows **Lo & MacKinlay (1988)** exactly: - -1. **Variance Ratio Calculation**: - - VR(q) = σ̂²_a(q) / σ̂²_b(q) - - σ̂²_a(q): Variance of non-overlapping q-period returns - - σ̂²_b(q): Weighted sum of autocovariances per equation (6a) - -2. **Test Statistic** (Homoskedastic): - - z(q) = M(q) × √(nq) / √θ(q) - - M(q) = VR(q) - 1 - - θ(q) = [2(2q-1)(q-1)] / (3q) - - Per equation (8) in Lo & MacKinlay (1988) - -3. **Test Statistic** (Heteroskedastic): - - z(q) = M(q) × √(nq) / √v̂(q) - - v̂(q): Asymptotic variance under heteroskedasticity - - Uses weighted delta statistics - -### Price Path Generation - -The `price_paths.py` module generates price trajectories for various stochastic processes: - -- **Geometric Brownian Motion (GBM)**: dS = μS dt + σS dW -- **Merton Jump Diffusion**: GBM with Poisson jumps -- **Heston Stochastic Volatility**: Mean-reverting volatility process -- **Vasicek Interest Rate Model**: Mean-reverting rates -- **Cox-Ingersoll-Ross (CIR)**: Square-root diffusion process -- **Ornstein-Uhlenbeck Process**: Mean-reverting process - -**Important**: All price path generators return **actual prices**, not log-prices. The simulation framework automatically converts actual prices to log-prices before applying the VRT. - -### Key Features - -- ✅ Correct implementation of Lo & MacKinlay (1988) formulas -- ✅ Both homoskedastic and heteroskedastic tests -- ✅ Proper handling of log-prices vs. actual prices -- ✅ Edge case handling for small samples -- ✅ Multiple stochastic process simulations -- ✅ Comprehensive testing framework - -## Mathematical Foundation - -### Variance Ratio Definition - -For a log price series X_t, the variance ratio at horizon q is: - -``` -VR(q) = Var[X_t - X_{t-q}] / (q × Var[X_t - X_{t-1}]) -``` - -Under the random walk null hypothesis, VR(q) should equal 1. - -### Weighted Autocovariance Formula - -The denominator σ̂²_b(q) is computed as: -``` -σ̂²_b(q) = Σ_{j=0}^{q-1} [2(q-j)/q] × γ̂(j) -``` - -where γ̂(j) is the j-th lag sample autocovariance with unbiased denominator (n-j). - -## Interpretation - -The Variance Ratio test results do not necessarily imply that the stock market is inefficient or that prices are not rational assessments of fundamental values. The test is purely a **descriptive tool** for examining the stochastic evolution of prices through time. - -- **VR(q) > 1**: Positive serial correlation (momentum) -- **VR(q) < 1**: Negative serial correlation (mean reversion) -- **VR(q) ≈ 1**: Consistent with random walk hypothesis - -### Statistical Significance +## Interpretation notes -- Use the z-statistic to test the null hypothesis that VR(q) = 1 -- Under the null, z follows a standard normal distribution -- Reject the null if |z| > 1.96 (5% significance level) +The battery provides evidence against compatibility with weak-form efficiency under this battery design. It is not definitive proof of market inefficiency. -## References +## Release status -### Primary Sources +This documentation corresponds to release `0.1.0`. The repository is prepared for a `0.1.0` publication workflow. Effective publication depends on the maintainer environment and credentials. -- Lo, Andrew W. and MacKinlay, Archie Craig, **Stock Market Prices Do Not Follow Random Walks: Evidence from a Simple Specification Test** (February 1987). NBER Working Paper No. w2168. Available at SSRN: https://ssrn.com/abstract=346975 +## Local release process -- Lo, Andrew W. and MacKinlay, Archie Craig, **The Size and Power of the Variance Ratio Test in Finite Samples: a Monte Carlo Investigation** (June 1988). NBER Working Paper No. t0066. Available at SSRN: https://ssrn.com/abstract=396681 +Manual maintainer commands for a local release flow: -### Additional Resources - -- Stuart Reid | On February. "Stock Market Prices Do Not Follow Random Walks." Turing Finance, 8 Feb. 2016, www.turingfinance.com/stock-market-prices-do-not-follow-random-walks/ - -- "Variance Ratio Test." Breaking Down Finance, [breakingdownfinance.com/finance-topics/finance-basics/variance-ratio-test/](breakingdownfinance.com/finance-topics/finance-basics/variance-ratio-test/) - -## Project Structure - -``` -variance-test/ -├── src/ -│ └── variance_test/ -│ ├── __init__.py -│ ├── core.py # Core VRT implementation (EMH class) -│ ├── battery.py # Weak-form battery v1 -│ ├── rolling.py # Internal rolling helpers -│ ├── data.py # Input normalization -│ ├── models.py # Dataclass contracts -│ ├── price_paths.py # Stochastic process generators -│ ├── simulation.py # Simulation framework and CLI -│ └── visuals.py # Plotting utilities -├── examples/ -│ ├── basic_battery.py # Offline battery example (full sample) -│ ├── rolling_battery.py # Offline battery example (rolling mode) -│ ├── variance_ratio_only.py # Offline EMH.vrt usage example -│ └── empirical_application.py # Optional external example -├── tests/ -├── pyproject.toml -├── README.md -└── LICENSE +```bash +python -m build +twine check dist/* +twine upload dist/* ``` -The historical internal module `variance_test.py` was renamed to `core.py` to avoid colliding with the package name. Import public symbols from `variance_test`. +These are manual maintainer steps and are not executed automatically by this repository. -## Recent Updates +## Development notes -### Version 2.0 (January 2026) - -**Critical Fixes to Variance Ratio Test Implementation**: - -1. ✅ Fixed inverted variance ratio formula (was σ_b²/σ_a², now σ_a²/σ_b²) -2. ✅ Corrected σ_b calculation using weighted autocovariances per Lo & MacKinlay equation (6a) -3. ✅ Fixed test statistic scaling (removed extra √q factor) -4. ✅ Fixed autocovariance denominators to use (n-j) per standard convention -5. ✅ Added edge case handling for boundary conditions -6. ✅ Fixed simulation.py to convert actual prices to log prices before VRT - -These fixes ensure the implementation exactly matches the Lo & MacKinlay (1988) methodology and produces statistically correct results. - -## Contributing - -Contributions are welcome! Please ensure any changes: -- Follow the Lo & MacKinlay (1988) methodology -- Include comprehensive tests -- Update documentation accordingly -- Pass all existing tests - -## Future Work - -- Distribution of the code as a PyPI package -- Step-by-step explanation for QuantConnect integration -- Implementation of Long Term Memory in Stock Market Prices (Lo, 1989) -- Additional market efficiency tests - -## License - -See LICENSE file for details. - -## Citation - -If you use this implementation in your research, please cite: - -```bibtex -@software{variance_ratio_test, - title = {Variance Ratio Test Implementation}, - author = {Parada, Lautaro}, - year = {2026}, - url = {https://github.com/LautaroParada/variance-test}, - note = {Implementation following Lo \& MacKinlay (1988)} -} -``` +- The test suite should pass before release. +- Examples and documentation should remain aligned with the public API.