Tic-Tac-Toe Q-Learning

A minimal, reproducible reinforcement learning project demonstrating tabular Q-Learning on Tic-Tac-Toe.
Designed for education, experimentation, and interactive play against a trained agent.

---

Features

• Fully reproducible training using configurable hyperparameters and random seeds
• Modular and clean code structure
• Interactive human vs AI gameplay with multiple difficulty levels
• Save/load Q-tables for reuse or evaluation
• Training progress visualization via win/draw/loss plots
• Live agent evaluation stats during human play
• Lightweight — pure NumPy, no external RL libraries required

---

Algorithm

This project implements a tabular Q-Learning agent for Tic-Tac-Toe.

The Q-Learning update rule:

Q(s,a) ← Q(s,a) + α [ r + γ max_a' Q(s',a') - Q(s,a) ]

Where:

• s = current state
• a = action taken
• r = reward received
• s' = next state
• α = learning rate
• γ = discount factor

The agent learns optimal moves by playing against a random opponent.

---

Installation

1. Clone the repository:

git clone https://github.com/yourusername/tic-tac-toe-qlearning.git
cd tic-tac-toe-qlearning

2. Install dependencies:

pip install -r requirements.txt

Dependencies: Python 3.8+, NumPy, Matplotlib

---

Usage

1. Train a Q-Learning Agent

python train.py --episodes 50000 --save q_table.pkl --plot training_plot.png

• Trains the agent against a random opponent.
• Saves the learned Q-table to q_table.pkl.
• Generates a plot of win/draw/loss ratios over training episodes.

2. Evaluate a Trained Agent

python train.py --eval --load q_table.pkl

• Runs multiple games against a random opponent.
• Prints win/draw/loss statistics.

3. Play Against the Agent (Interactive)

python play.py --level hard --load q_table.pkl --eval_games 200

• --level options:

  • easy → agent with high exploration (ε ≈ 0.5)
  • medium → agent with moderate exploration (ε ≈ 0.1)
  • hard → fully trained agent (ε ≈ 0.01)

• CLI will prompt for moves and display the board after each turn.

• Displays live evaluation stats of the agent’s performance before and after your game.

---

Configuration

All hyperparameters are configurable in config.yaml:

seed: 42
episodes: 50000
learning_rate: 0.2
gamma: 0.95
epsilon_start: 1.0
epsilon_min: 0.02
epsilon_decay: 0.99998
eval_interval: 5000

• Adjust episodes for longer or shorter training.
• epsilon_* values control exploration/exploitation balance.

---