DS.v2.5.3.4.5

NVDA Market Regimes & Reinforcement Learning

Project Overview

This project explores how unsupervised learning (market regimes) can be combined with reinforcement learning (RL) to model trading decisions on NVIDIA (NVDA) daily stock data.

The goal is not to beat buy-and-hold, but to:

• understand market structure,
• compare risk-aware vs profit-seeking policies,
• evaluate realistic trading performance.

---

Project Structure

src/
├── functions.py          # Data processing, EDA, regimes, RL logic
├── notebook.ipynb        # End-to-end analysis & experiments
├── nvda_market_data.csv  # NVDA daily data (Yahoo Finance)
├── requirements.txt      # Project dependencies
└── workspace.code-workspace

---

Data

• Asset: NVIDIA (NVDA)
• Source: Yahoo Finance (yfinance)
• Frequency: Daily
• Period: 2010 – Present
• Features: OHLCV + engineered features

---

Feature Engineering

• ret – daily return (%)
• volatility – rolling 20-day return standard deviation (%)
• volume_chg – daily volume change (%)

These describe price movement, risk, and market participation.

---

Exploratory Data Analysis (EDA)

• Heavy-tailed returns and volume
• Volatility clustering
• Extreme return outliers (±20%)
• Weak linear correlations → motivates non-linear models

---

Unsupervised Learning: Market Regimes

• Method: K-Means clustering
• Inputs: return & volatility
• Identified regimes:
  • Bullish (positive returns)
  • Bearish (negative returns)
  • Sideways / low-volatility

Regimes provide market context, not direct predictions.

---

Reinforcement Learning

• Environment: NVDA daily returns with transaction costs
• Actions: {-1 = short, 0 = hold, +1 = long}
• Agent: Tabular Q-Learning
• Training: ε-greedy exploration (~20%, decayed)
• Evaluation: greedy policy (ε = 0)

Two agents are compared:

• Regime-blind RL
• Regime-aware RL (regime included in state)

---

Evaluation Metrics

• Cumulative PnL
• Average return
• Max drawdown (downside risk)
• Hit ratio (timing accuracy)
• Turnover (trading frequency)

---

Key Findings

• Buy-and-hold dominates in a strong bull market (NVDA).
• Regime-blind RL is more aggressive and trend-driven.
• Regime-aware RL trades less, with improved timing and risk control.
• Regimes add risk awareness, not guaranteed alpha.
• RL results vary due to stochastic exploration.

---

Limitations

• Single-asset setting
• Simple regime definition
• Tabular RL
• In-sample evaluation only

---

Future Improvements

• Multi-asset portfolios
• HMM or macro-based regimes
• Deep RL (DQN, Actor-Critic)
• Risk-adjusted rewards
• Walk-forward / out-of-sample testing

---

Setup

pip install -r requirements.txt

---

Disclaimer

This project is for educational purposes only
and does not constitute financial advice.

Author: DS AI learner Karolis Vaivada