Production Testing Updates & Fixes

.vscode/settings.json

@@ -0,0 +1,8 @@
+{
+  "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
+  "python.analysis.extraPaths": ["${workspaceFolder}/src"],
+  "python.terminal.activateEnvironment": true,
+  "python.analysis.diagnosticMode": "workspace",
+  "cursorpyright.analysis.venvPath": "${workspaceFolder}",
+  "cursorpyright.analysis.venv": ".venv"
+}

README.md

@@ -1,149 +1,134 @@
-# Poker AI Training Pipeline
+# Poker AI
 
-A machine learning pipeline for training a GPT-2 model to predict poker actions from game states. The project uses transformer-based language modeling to learn poker decision-making from hand history data.
-
-## Project Overview
-
-This project trains a neural network to predict poker actions (FOLD, CALL, RAISE) given the current game state. The model learns from real poker hand histories using a masked language modeling approach where the context (game state) is provided and the model must predict the action.
+Train a small GPT-2 decoder to predict poker actions (FOLD, CALL, RAISE, …) from a serialized game state. Loss is applied only to the action tokens (completion-only / masked training).
 
 ## Setup
 
-1. **Clone the repository**
-   ```bash
-   git clone https://github.com/YOUR_USERNAME/pokerAI.git
-   cd pokerAI
-   ```
-
-2. **Create virtual environment**
-   ```bash
-   python -m venv venv
-   venv\Scripts\activate  # On Windows
-   source venv/bin/activate  # On Linux/Mac
-   ```
-
-3. **Install dependencies**
-   ```bash
-   pip install -r requirements.txt
-   ```
-
-4. **Prepare your data**
-   - Place your poker hand history data in `data/hands.txt`
-   - Each line should be a complete hand history in the format: `POSITION,STACK,CARDS,PLAYER_INFO,ACTION`
-
-## Training Pipeline
-
-### 1. Data Preparation (`data/prepare_data.py`)
-Cleans the raw poker data and creates train/test splits:
-- Removes redundant "0BB" from FOLD actions
-- Splits data into 95% training / 5% validation
-- Saves cleaned data to `data/hands_fold_update.txt`
-
 ```bash
-python data/prepare_data.py
+git clone https://github.com/Exios66/pokerai.git
+cd pokerai
+python -m venv .venv
+source .venv/bin/activate          # Windows: .venv\Scripts\activate
+pip install -e ".[dev]"            # or: pip install -r requirements.txt && pip install -e .
 ```
 
-### 2. Tokenizer Training (`data/train_tokenizer.py`)
-Trains a custom BPE tokenizer on the poker domain:
-- Vocabulary size: 4000 tokens
-- Special tokens: `<|startoftext|>`, `<|pad|>`
-- Saves tokenizer to `tokenizer/` directory
+Weights & Biases is **optional**. Training runs without it by default. To enable logging:
 
 ```bash
-python data/train_tokenizer.py
+wandb login
+# or offline: export WANDB_MODE=offline
 ```
 
-### 3. Baseline Model (`data/train_bigram.py`)
-Trains a simple bigram model for comparison:
-- Provides a baseline for model performance
-- Expected validation loss: ~1.53
+To force-disable: `export WANDB_MODE=disabled`.
 
-```bash
-python data/train_bigram.py
-```
+## Pipeline
 
-### 4. GPT-2 Model Training
+| Step | Command | Output |
+|------|---------|--------|
+| 1. Fetch & clean data | `python scripts/prepare_data.py` | `data/hands.txt`, `data/hands_clean.txt` |
+| 2. Train tokenizer | `python scripts/train_tokenizer.py` | `artifacts/tokenizer/` |
+| 3. Bigram baseline | `python scripts/train_bigram.py` | `artifacts/models/bigram/` |
+| 4a. GPT-2 (raw loop) | `python scripts/train_gpt2.py` | `artifacts/models/gpt2/` |
+| 4b. GPT-2 (TRL) | `python scripts/train_trl.py` | `artifacts/models/gpt2_trl/` |
+| 5. Predict | `python scripts/predict.py "<state>"` | printed action |
 
-**Option A: Raw PyTorch Loop (`data/build_model.py`)**
-- GPT-2 architecture (6 layers, 256 hidden dimensions, 8 heads)
-- Context length: 192 tokens
-- Masked training: only predicts the action, not the context
-- 3 epochs with AdamW optimizer (lr=3e-4)
+After `pip install -e .` you can also use `pokerai-prepare`, `pokerai-gpt2`, `pokerai-predict`, etc.
 
-```bash
-python data/build_model.py
-```
+**Optional:** `python scripts/convert_poker_dataset.py` also downloads the HF dataset, writes the same `hands.txt` / `hands_clean.txt`, and additionally preserves the official HF train/test split as `data/hands_train.txt` and `data/hands_test.txt`. Trainers still load `hands_clean.txt` by default (random 95/5 split).
 
-**Option B: TRL Trainer (`data/TRL_model.py`)**
-- Uses Hugging Face TRL library for supervised fine-tuning
-- Same model architecture as raw loop
-- Prompt/completion format for masked training
-- Saves model to `model_out_trl/`
+## Data
+
+Hands are downloaded from Hugging Face [`SoelMgd/Poker_Dataset`](https://huggingface.co/datasets/SoelMgd/Poker_Dataset). Each line is `context,action` — split at the **last comma**.
+
+Example:
 
-```bash
-python data/TRL_model.py
 ```
+[TABLE_CONFIGURATION] BTN=P3 SB=P1 0.5BB BB=P2 1BB [STACKS] P1: 44.2BB [Qh 9h] P2: 103.4BB P3: 165.2BB POT=1.5BB [PREFLOP] P3: RAISE 2BB P1:,FOLD
+```
+
+Cleaning normalizes redundant suffixes (`CALL 0BB` → `CALL`, legacy `FOLD0BB` → `FOLD`).
 
-## Model Architecture
+The tokenizer is trained on **cleaned** hands (`data/hands_clean.txt`) so vocabulary matches what models see at train time.
 
-- **Model Type**: GPT-2 (decoder-only transformer)
-- **Parameters**: ~2.6M
-- **Layers**: 6
-- **Hidden Size**: 256
-- **Attention Heads**: 8
-- **Context Window**: 192 tokens
-- **Vocabulary Size**: 4000 (domain-specific)
+## Model
 
-## Training Strategy
+| Setting | Value |
+|---------|-------|
+| Architecture | GPT-2 (random init) |
+| Layers / hidden / heads | 6 / 256 / 8 |
+| Context length | **384** tokens |
+| Vocab | Domain BPE (target size 4000; actual size depends on data) |
+| Special tokens | `<\|startoftext\|>` (BOS), `<\|endoftext\|>` (EOS), `<\|pad\|>` |
+| Optimizer | AdamW, lr `3e-4`, 3 epochs, batch 32 |
 
 The model uses masked language modeling:
-- **Input**: Full hand history (game state + action)
-- **Masking**: All tokens up to the last comma (the game state) are masked
-- **Prediction**: Model learns to predict only the action tokens
-- This ensures the model focuses on decision-making, not memorizing game states
 
-## Data Format
+- **Input**: Full hand history (game state + action), with BOS/EOS
+- **Masking**: All tokens up to the decision separator (the game state) are masked (`-100`)
+- **Prediction**: Model learns to predict only the action tokens (+ EOS)
+- Long sequences **keep the end** (action), dropping tokens from the front
 
-Each hand history line should follow this format:
-```
-POSITION,STACK,CARDS,PLAYER1_INFO,PLAYER2_INFO,...,ACTION
-```
+**Implementation detail:** the prompt/answer boundary is located by character offset in the full tokenized sequence (via `return_offsets_mapping`), not by tokenizing the prompt separately and re-joining. BPE merge behavior can differ depending on what follows a substring, so tokenizing the prompt in isolation can silently misalign the mask.
+
+`<|startoftext|>` and `<|endoftext|>` must be distinct tokens. If BOS and EOS are the same token, the model can't tell "hand starting" from "hand ending."
+
+## Layout
 
-Example:
 ```
-BTN,1.5BB,9d 10c,P1:101.0BB/0.0BB,P2:100.23BB/0.0BB,FOLD
+pokerai/
+├── src/pokerai/           # installable package
+│   ├── config.py          # paths + hyperparameters
+│   ├── data/              # fetch, clean, tokenize, encode/mask
+│   ├── models/            # Bigram + GPT-2 factory
+│   ├── training/          # bigram / gpt2 / trl trainers
+│   └── inference/         # action prediction
+├── scripts/               # thin CLI wrappers
+├── tests/
+├── data/                  # hand text only (gitignored)
+└── artifacts/             # tokenizer + models (gitignored)
 ```
 
-## File Structure
+## Tracking runs with W&B
 
-```
-pokerAI/
-├── data/
-│   ├── prepare_data.py      # Data cleaning and splitting
-│   ├── train_tokenizer.py   # Custom tokenizer training
-│   ├── train_bigram.py       # Baseline model training
-│   ├── build_model.py       # GPT-2 training (raw loop)
-│   ├── TRL_model.py         # GPT-2 training (TRL)
-│   ├── hands.txt            # Raw input data (not in repo)
-│   └── hands_fold_update.txt # Cleaned data (not in repo)
-├── tokenizer/                # Trained tokenizer (not in repo)
-├── model_out_trl/           # Trained model (not in repo)
-├── requirements.txt         # Python dependencies
-└── README.md                # This file
+When configured, all training scripts log to
+[`mooslin-university-of-wisconsin-madison/poker-ai`](https://wandb.ai/mooslin-university-of-wisconsin-madison/poker-ai)
+so runs are comparable. TRL uses `TrainingArguments(report_to=...)`; the raw-loop scripts call `wandb.init` / `wandb.log` only when W&B is available.
+
+On offline compute nodes:
+
+```bash
+export WANDB_MODE=offline
+python scripts/train_gpt2.py
+wandb sync wandb/offline-run-*
 ```
 
-## Performance
+## Experiment tracking
 
 - **Bigram baseline**: Validation loss ~1.53
 - **GPT-2 model**: Expected to significantly outperform baseline
 - Compare final validation loss against baseline to assess improvement
+- Check the action-type distribution before over-interpreting loss — poker decision data is typically FOLD-heavy, so raw loss alone can hide poor performance on rarer actions like RAISE
+
+**Full experiment catalog** (configs, charts, action metrics, alternative approaches): see [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md).
+
+```bash
+python scripts/run_experiment.py majority --group imbalance --tags exp-f2
+python scripts/run_experiment.py features --method rf --group alt-approaches
+python scripts/run_experiment.py weighted-gpt2 --group imbalance --tags exp-f3
+python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256
+```
+
+Add `--require-wandb` for catalog/sweep runs that must log. Sweeps: [`docs/wandb_sweep_gpt2_capacity.yaml`](docs/wandb_sweep_gpt2_capacity.yaml), [`docs/wandb_sweep_model_compare.yaml`](docs/wandb_sweep_model_compare.yaml).
 
 ## Notes
 
-- Large files (data, models, tokenizer) are excluded from git via .gitignore
-- These can be regenerated by running the training scripts
-- Model training requires CUDA GPU for reasonable speed
+- Large files (data, models, tokenizer) are excluded from git via `.gitignore` and can be regenerated from the scripts above
+- Model training is much faster with a CUDA GPU; TRL and the raw loop also run on CPU
 - Adjust batch size and learning rate based on your hardware
+- The bigram baseline is a sanity check — if GPT-2 does not clearly beat it, something upstream (tokenizer, data format, masking) is wrong
 
-## License
+## Known limitations
 
-[Add your license here]
+- Class imbalance is partially addressed via `majority`, `features` (balanced), and `weighted-gpt2`; no focal loss yet
+- No checkpoint resumption during training
+- Condensed shorthand notation for the POC dataset is not yet implemented — converters pass the (flattened) raw text through