diff --git a/.gitignore b/.gitignore index b0614fc..3b59818 100644 Binary files a/.gitignore and b/.gitignore differ diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 0000000..b0cef7d --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,8 @@ +{ + "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python", + "python.analysis.extraPaths": ["${workspaceFolder}/src"], + "python.terminal.activateEnvironment": true, + "python.analysis.diagnosticMode": "workspace", + "cursorpyright.analysis.venvPath": "${workspaceFolder}", + "cursorpyright.analysis.venv": ".venv" +} diff --git a/README.md b/README.md index 229fd06..9865f32 100644 --- a/README.md +++ b/README.md @@ -1,149 +1,134 @@ -# Poker AI Training Pipeline +# Poker AI -A machine learning pipeline for training a GPT-2 model to predict poker actions from game states. The project uses transformer-based language modeling to learn poker decision-making from hand history data. - -## Project Overview - -This project trains a neural network to predict poker actions (FOLD, CALL, RAISE) given the current game state. The model learns from real poker hand histories using a masked language modeling approach where the context (game state) is provided and the model must predict the action. +Train a small GPT-2 decoder to predict poker actions (FOLD, CALL, RAISE, …) from a serialized game state. Loss is applied only to the action tokens (completion-only / masked training). ## Setup -1. **Clone the repository** - ```bash - git clone https://github.com/YOUR_USERNAME/pokerAI.git - cd pokerAI - ``` - -2. **Create virtual environment** - ```bash - python -m venv venv - venv\Scripts\activate # On Windows - source venv/bin/activate # On Linux/Mac - ``` - -3. **Install dependencies** - ```bash - pip install -r requirements.txt - ``` - -4. **Prepare your data** - - Place your poker hand history data in `data/hands.txt` - - Each line should be a complete hand history in the format: `POSITION,STACK,CARDS,PLAYER_INFO,ACTION` - -## Training Pipeline - -### 1. Data Preparation (`data/prepare_data.py`) -Cleans the raw poker data and creates train/test splits: -- Removes redundant "0BB" from FOLD actions -- Splits data into 95% training / 5% validation -- Saves cleaned data to `data/hands_fold_update.txt` - ```bash -python data/prepare_data.py +git clone https://github.com/Exios66/pokerai.git +cd pokerai +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -e ".[dev]" # or: pip install -r requirements.txt && pip install -e . ``` -### 2. Tokenizer Training (`data/train_tokenizer.py`) -Trains a custom BPE tokenizer on the poker domain: -- Vocabulary size: 4000 tokens -- Special tokens: `<|startoftext|>`, `<|pad|>` -- Saves tokenizer to `tokenizer/` directory +Weights & Biases is **optional**. Training runs without it by default. To enable logging: ```bash -python data/train_tokenizer.py +wandb login +# or offline: export WANDB_MODE=offline ``` -### 3. Baseline Model (`data/train_bigram.py`) -Trains a simple bigram model for comparison: -- Provides a baseline for model performance -- Expected validation loss: ~1.53 +To force-disable: `export WANDB_MODE=disabled`. -```bash -python data/train_bigram.py -``` +## Pipeline -### 4. GPT-2 Model Training +| Step | Command | Output | +|------|---------|--------| +| 1. Fetch & clean data | `python scripts/prepare_data.py` | `data/hands.txt`, `data/hands_clean.txt` | +| 2. Train tokenizer | `python scripts/train_tokenizer.py` | `artifacts/tokenizer/` | +| 3. Bigram baseline | `python scripts/train_bigram.py` | `artifacts/models/bigram/` | +| 4a. GPT-2 (raw loop) | `python scripts/train_gpt2.py` | `artifacts/models/gpt2/` | +| 4b. GPT-2 (TRL) | `python scripts/train_trl.py` | `artifacts/models/gpt2_trl/` | +| 5. Predict | `python scripts/predict.py ""` | printed action | -**Option A: Raw PyTorch Loop (`data/build_model.py`)** -- GPT-2 architecture (6 layers, 256 hidden dimensions, 8 heads) -- Context length: 192 tokens -- Masked training: only predicts the action, not the context -- 3 epochs with AdamW optimizer (lr=3e-4) +After `pip install -e .` you can also use `pokerai-prepare`, `pokerai-gpt2`, `pokerai-predict`, etc. -```bash -python data/build_model.py -``` +**Optional:** `python scripts/convert_poker_dataset.py` also downloads the HF dataset, writes the same `hands.txt` / `hands_clean.txt`, and additionally preserves the official HF train/test split as `data/hands_train.txt` and `data/hands_test.txt`. Trainers still load `hands_clean.txt` by default (random 95/5 split). -**Option B: TRL Trainer (`data/TRL_model.py`)** -- Uses Hugging Face TRL library for supervised fine-tuning -- Same model architecture as raw loop -- Prompt/completion format for masked training -- Saves model to `model_out_trl/` +## Data + +Hands are downloaded from Hugging Face [`SoelMgd/Poker_Dataset`](https://huggingface.co/datasets/SoelMgd/Poker_Dataset). Each line is `context,action` — split at the **last comma**. + +Example: -```bash -python data/TRL_model.py ``` +[TABLE_CONFIGURATION] BTN=P3 SB=P1 0.5BB BB=P2 1BB [STACKS] P1: 44.2BB [Qh 9h] P2: 103.4BB P3: 165.2BB POT=1.5BB [PREFLOP] P3: RAISE 2BB P1:,FOLD +``` + +Cleaning normalizes redundant suffixes (`CALL 0BB` → `CALL`, legacy `FOLD0BB` → `FOLD`). -## Model Architecture +The tokenizer is trained on **cleaned** hands (`data/hands_clean.txt`) so vocabulary matches what models see at train time. -- **Model Type**: GPT-2 (decoder-only transformer) -- **Parameters**: ~2.6M -- **Layers**: 6 -- **Hidden Size**: 256 -- **Attention Heads**: 8 -- **Context Window**: 192 tokens -- **Vocabulary Size**: 4000 (domain-specific) +## Model -## Training Strategy +| Setting | Value | +|---------|-------| +| Architecture | GPT-2 (random init) | +| Layers / hidden / heads | 6 / 256 / 8 | +| Context length | **384** tokens | +| Vocab | Domain BPE (target size 4000; actual size depends on data) | +| Special tokens | `<\|startoftext\|>` (BOS), `<\|endoftext\|>` (EOS), `<\|pad\|>` | +| Optimizer | AdamW, lr `3e-4`, 3 epochs, batch 32 | The model uses masked language modeling: -- **Input**: Full hand history (game state + action) -- **Masking**: All tokens up to the last comma (the game state) are masked -- **Prediction**: Model learns to predict only the action tokens -- This ensures the model focuses on decision-making, not memorizing game states -## Data Format +- **Input**: Full hand history (game state + action), with BOS/EOS +- **Masking**: All tokens up to the decision separator (the game state) are masked (`-100`) +- **Prediction**: Model learns to predict only the action tokens (+ EOS) +- Long sequences **keep the end** (action), dropping tokens from the front -Each hand history line should follow this format: -``` -POSITION,STACK,CARDS,PLAYER1_INFO,PLAYER2_INFO,...,ACTION -``` +**Implementation detail:** the prompt/answer boundary is located by character offset in the full tokenized sequence (via `return_offsets_mapping`), not by tokenizing the prompt separately and re-joining. BPE merge behavior can differ depending on what follows a substring, so tokenizing the prompt in isolation can silently misalign the mask. + +`<|startoftext|>` and `<|endoftext|>` must be distinct tokens. If BOS and EOS are the same token, the model can't tell "hand starting" from "hand ending." + +## Layout -Example: ``` -BTN,1.5BB,9d 10c,P1:101.0BB/0.0BB,P2:100.23BB/0.0BB,FOLD +pokerai/ +├── src/pokerai/ # installable package +│ ├── config.py # paths + hyperparameters +│ ├── data/ # fetch, clean, tokenize, encode/mask +│ ├── models/ # Bigram + GPT-2 factory +│ ├── training/ # bigram / gpt2 / trl trainers +│ └── inference/ # action prediction +├── scripts/ # thin CLI wrappers +├── tests/ +├── data/ # hand text only (gitignored) +└── artifacts/ # tokenizer + models (gitignored) ``` -## File Structure +## Tracking runs with W&B -``` -pokerAI/ -├── data/ -│ ├── prepare_data.py # Data cleaning and splitting -│ ├── train_tokenizer.py # Custom tokenizer training -│ ├── train_bigram.py # Baseline model training -│ ├── build_model.py # GPT-2 training (raw loop) -│ ├── TRL_model.py # GPT-2 training (TRL) -│ ├── hands.txt # Raw input data (not in repo) -│ └── hands_fold_update.txt # Cleaned data (not in repo) -├── tokenizer/ # Trained tokenizer (not in repo) -├── model_out_trl/ # Trained model (not in repo) -├── requirements.txt # Python dependencies -└── README.md # This file +When configured, all training scripts log to +[`mooslin-university-of-wisconsin-madison/poker-ai`](https://wandb.ai/mooslin-university-of-wisconsin-madison/poker-ai) +so runs are comparable. TRL uses `TrainingArguments(report_to=...)`; the raw-loop scripts call `wandb.init` / `wandb.log` only when W&B is available. + +On offline compute nodes: + +```bash +export WANDB_MODE=offline +python scripts/train_gpt2.py +wandb sync wandb/offline-run-* ``` -## Performance +## Experiment tracking - **Bigram baseline**: Validation loss ~1.53 - **GPT-2 model**: Expected to significantly outperform baseline - Compare final validation loss against baseline to assess improvement +- Check the action-type distribution before over-interpreting loss — poker decision data is typically FOLD-heavy, so raw loss alone can hide poor performance on rarer actions like RAISE + +**Full experiment catalog** (configs, charts, action metrics, alternative approaches): see [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md). + +```bash +python scripts/run_experiment.py majority --group imbalance --tags exp-f2 +python scripts/run_experiment.py features --method rf --group alt-approaches +python scripts/run_experiment.py weighted-gpt2 --group imbalance --tags exp-f3 +python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256 +``` + +Add `--require-wandb` for catalog/sweep runs that must log. Sweeps: [`docs/wandb_sweep_gpt2_capacity.yaml`](docs/wandb_sweep_gpt2_capacity.yaml), [`docs/wandb_sweep_model_compare.yaml`](docs/wandb_sweep_model_compare.yaml). ## Notes -- Large files (data, models, tokenizer) are excluded from git via .gitignore -- These can be regenerated by running the training scripts -- Model training requires CUDA GPU for reasonable speed +- Large files (data, models, tokenizer) are excluded from git via `.gitignore` and can be regenerated from the scripts above +- Model training is much faster with a CUDA GPU; TRL and the raw loop also run on CPU - Adjust batch size and learning rate based on your hardware +- The bigram baseline is a sanity check — if GPT-2 does not clearly beat it, something upstream (tokenizer, data format, masking) is wrong -## License +## Known limitations -[Add your license here] +- Class imbalance is partially addressed via `majority`, `features` (balanced), and `weighted-gpt2`; no focal loss yet +- No checkpoint resumption during training +- Condensed shorthand notation for the POC dataset is not yet implemented — converters pass the (flattened) raw text through diff --git a/data/.gitkeep b/data/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/data/TRL_model.py b/data/TRL_model.py deleted file mode 100644 index 32c43ed..0000000 --- a/data/TRL_model.py +++ /dev/null @@ -1,62 +0,0 @@ -import torch -from datasets import load_dataset -from transformers import GPT2TokenizerFast, GPT2Config, GPT2LMHeadModel -from trl import SFTConfig, SFTTrainer - -tokenizer = GPT2TokenizerFast.from_pretrained("tokenizer") - -# Same random-init model as the raw loop -- from_config, not from_pretrained. -config = GPT2Config( - vocab_size=len(tokenizer), - n_positions=192, - n_embd=256, - n_layer=6, - n_head=8, - bos_token_id=tokenizer.bos_token_id, - eos_token_id=tokenizer.eos_token_id, - pad_token_id=tokenizer.pad_token_id, -) -model = GPT2LMHeadModel(config) -print(f"Model has {sum(p.numel() for p in model.parameters()):,} parameters") - -# Reshape into prompt/completion pairs at the last comma -- TRL masks -# the prompt and computes loss on the completion by default. -def split_prompt_completion(example): - text = example["text"] - cut = text.rfind(",") + 1 - return {"prompt": text[:cut], "completion": text[cut:]} - -raw = load_dataset("text", data_files={"train": "data/hands_fold_update.txt"})["train"] -raw = raw.train_test_split(test_size=0.05, seed=42) -train_dataset = raw["train"].map(split_prompt_completion, remove_columns=["text"]) -eval_dataset = raw["test"].map(split_prompt_completion, remove_columns=["text"]) -print(train_dataset[0]) - -training_args = SFTConfig( - output_dir="model_out_trl", - num_train_epochs=3, - per_device_train_batch_size=32, - per_device_eval_batch_size=32, - eval_strategy="epoch", - logging_steps=50, - learning_rate=3e-4, - max_length=192, - completion_only_loss=True, # default for prompt/completion data, explicit here for clarity - fp16=torch.cuda.is_available(), - report_to="none", -) - -trainer = SFTTrainer( - model=model, - args=training_args, - train_dataset=train_dataset, - eval_dataset=eval_dataset, - processing_class=tokenizer, -) - -trainer.train() -trainer.save_model("model_out_trl") -tokenizer.save_pretrained("model_out_trl") -print("Saved to model_out_trl/") - - diff --git a/data/build_model.py b/data/build_model.py deleted file mode 100644 index 8e0979a..0000000 --- a/data/build_model.py +++ /dev/null @@ -1,113 +0,0 @@ -import torch -import torch.nn.functional as F -from torch.utils.data import Dataset, DataLoader -from transformers import GPT2TokenizerFast, GPT2Config, GPT2LMHeadModel -from datasets import load_dataset - -tokenizer = GPT2TokenizerFast.from_pretrained("tokenizer") -vocab_size = len(tokenizer) - -# ---- Model: random init (from_config, not from_pretrained) ---- -config = GPT2Config( - vocab_size=vocab_size, - n_positions=192, - n_embd=256, - n_layer=6, - n_head=8, - bos_token_id=tokenizer.bos_token_id, - eos_token_id=tokenizer.eos_token_id, - pad_token_id=tokenizer.pad_token_id, -) -model = GPT2LMHeadModel(config) -print(f"Model has {sum(p.numel() for p in model.parameters()):,} parameters") - -device = "cuda" if torch.cuda.is_available() else "cpu" -model.to(device) -print("Training on:", device) - -# ---- Data: tokenize full sequence, mask everything up to the last comma ---- -raw = load_dataset("text", data_files={"train": "data/hands_fold_update.txt"})["train"] -raw = raw.train_test_split(test_size=0.05, seed=42) - -def encode_with_mask(text): - last_comma = text.rfind(",") - prompt_text = text[:last_comma + 1] # context (state), up to and including last comma - - prompt_ids = tokenizer(prompt_text)["input_ids"] - full_ids = [tokenizer.bos_token_id] + tokenizer(text)["input_ids"] + [tokenizer.eos_token_id] - prompt_len = len(prompt_ids) + 1 # +1 for BOS - - labels = [-100] * len(full_ids) - for i in range(prompt_len, len(full_ids)): - labels[i] = full_ids[i] - return full_ids, labels - -class HandsDataset(Dataset): - def __init__(self, split): - self.examples = [encode_with_mask(ex["text"]) for ex in split] - def __len__(self): - return len(self.examples) - def __getitem__(self, idx): - return self.examples[idx] - -def collate(batch): - max_len = max(len(ids) for ids, _ in batch) - pad_id = tokenizer.pad_token_id - input_ids, labels, attn_mask = [], [], [] - for ids, lbls in batch: - pad_len = max_len - len(ids) - input_ids.append(ids + [pad_id] * pad_len) - labels.append(lbls + [-100] * pad_len) - attn_mask.append([1] * len(ids) + [0] * pad_len) - return torch.tensor(input_ids), torch.tensor(labels), torch.tensor(attn_mask) - -train_ds = HandsDataset(raw["train"]) -val_ds = HandsDataset(raw["test"]) -train_loader = DataLoader(train_ds, batch_size=32, shuffle=True, collate_fn=collate) -val_loader = DataLoader(val_ds, batch_size=32, shuffle=False, collate_fn=collate) - -# ---- Sanity-check the masking on one real example ---- -ids, labels = train_ds[0] -print("\nTokens: ", tokenizer.convert_ids_to_tokens(ids)) -print("Labels: ", [tokenizer.convert_ids_to_tokens([t])[0] if t != -100 else "---" for t in labels]) -print("(Only non-'---' positions contribute to the loss -- verify that's just the decision.)\n") - -# ---- Raw training loop ---- -optimizer = torch.optim.AdamW(model.parameters(), lr=3e-4) - -def run_eval(): - model.eval() - total_loss, total_tokens = 0.0, 0 - with torch.no_grad(): - for input_ids, labels, attn_mask in val_loader: - input_ids, labels, attn_mask = input_ids.to(device), labels.to(device), attn_mask.to(device) - logits = model(input_ids=input_ids, attention_mask=attn_mask).logits[:, :-1, :].contiguous() - targets = labels[:, 1:].contiguous() - loss = F.cross_entropy(logits.reshape(-1, vocab_size), targets.reshape(-1), ignore_index=-100) - n_tok = (targets != -100).sum().item() - total_loss += loss.item() * n_tok - total_tokens += n_tok - model.train() - return total_loss / total_tokens - -step = 0 -for epoch in range(3): - for input_ids, labels, attn_mask in train_loader: - input_ids, labels, attn_mask = input_ids.to(device), labels.to(device), attn_mask.to(device) - logits = model(input_ids=input_ids, attention_mask=attn_mask).logits[:, :-1, :].contiguous() - targets = labels[:, 1:].contiguous() - loss = F.cross_entropy(logits.reshape(-1, vocab_size), targets.reshape(-1), ignore_index=-100) - - optimizer.zero_grad() - loss.backward() - optimizer.step() - - if step % 50 == 0: - print(f"epoch {epoch} step {step:5d} | train loss {loss.item():.4f}") - step += 1 - - print(f"== end of epoch {epoch}: val loss {run_eval():.4f} ==") - -print("\nBigram baseline val loss was ~1.53 -- compare final val loss above to that.") - - diff --git a/data/prepare_data.py b/data/prepare_data.py deleted file mode 100644 index b776223..0000000 --- a/data/prepare_data.py +++ /dev/null @@ -1,33 +0,0 @@ -import re -from datasets import load_dataset - -# Clean the data: remove 0BB from FOLD actions -with open("data/hands.txt", encoding="utf-8") as f: - lines = [l.rstrip("\n") for l in f] - -def clean(line): - # Strip the trailing amount only off FOLD -- it's always 0, pure noise. - return re.sub(r"FOLD\d+(\.\d+)?BB$", "FOLD", line) - -cleaned = [clean(l) for l in lines] - -changed = sum(1 for a, b in zip(lines, cleaned) if a != b) -print(f"Modified {changed} / {len(lines)} lines") -print("Before:", lines[0]) -print("After: ", cleaned[0]) - -# Save cleaned data -with open("data/hands_fold_update.txt", "w", encoding="utf-8") as f: - f.write("\n".join(cleaned) + "\n") - -# Load and split the cleaned dataset -dataset = load_dataset("text", data_files={"train": "data/hands_fold_update.txt"}) -dataset = dataset["train"].train_test_split(test_size=0.05, seed=42) - -print("\nDataset split:") -print(dataset) -print("Example hand:", dataset["train"][0]["text"]) -print("Total train examples:", len(dataset["train"])) -print("Total eval examples:", len(dataset["test"])) - - diff --git a/data/train_bigram.py b/data/train_bigram.py deleted file mode 100644 index 081a458..0000000 --- a/data/train_bigram.py +++ /dev/null @@ -1,74 +0,0 @@ -import math -import torch -import torch.nn as nn -import torch.nn.functional as F -from transformers import GPT2TokenizerFast -from datasets import load_dataset - -tokenizer = GPT2TokenizerFast.from_pretrained("tokenizer") -vocab_size = len(tokenizer) - -dataset = load_dataset("text", data_files={"train": "data/hands_fold_update.txt"})["train"] -dataset = dataset.train_test_split(test_size=0.05, seed=42) - -def encode(example): - ids = tokenizer(example["text"])["input_ids"] - return [tokenizer.bos_token_id] + ids + [tokenizer.eos_token_id] - -train_ids = [encode(ex) for ex in dataset["train"]] -val_ids = [encode(ex) for ex in dataset["test"]] - -# Turn each sequence into (current_token, next_token) pairs. -def make_pairs(sequences): - xs, ys = [], [] - for seq in sequences: - for i in range(len(seq) - 1): - xs.append(seq[i]) - ys.append(seq[i + 1]) - return torch.tensor(xs), torch.tensor(ys) - -train_x, train_y = make_pairs(train_ids) -val_x, val_y = make_pairs(val_ids) -print(f"Train pairs: {len(train_x):,} Val pairs: {len(val_x):,}") - -class Bigram(nn.Module): - def __init__(self, vocab_size): - super().__init__() - self.table = nn.Embedding(vocab_size, vocab_size) # row i = logits for "next token after i" - - def forward(self, idx): - return self.table(idx) - -device = "cuda" if torch.cuda.is_available() else "cpu" -print("Training on:", device) - -model = Bigram(vocab_size).to(device) -optimizer = torch.optim.AdamW(model.parameters(), lr=1e-2) - -train_x, train_y = train_x.to(device), train_y.to(device) -val_x, val_y = val_x.to(device), val_y.to(device) - -batch_size = 4096 -n_steps = 2000 - -for step in range(n_steps): - idx = torch.randint(0, len(train_x), (batch_size,)) - xb, yb = train_x[idx], train_y[idx] - - logits = model(xb) - loss = F.cross_entropy(logits, yb) - - optimizer.zero_grad() - loss.backward() - optimizer.step() - - if step % 200 == 0 or step == n_steps - 1: - with torch.no_grad(): - val_loss = F.cross_entropy(model(val_x), val_y) - print(f"step {step:5d} | train loss {loss.item():.4f} | val loss {val_loss.item():.4f}") - -random_baseline = math.log(vocab_size) -print(f"\nRandom-guess baseline loss: {random_baseline:.4f} (ln(vocab_size))") -print(f"Bigram model final val loss: {val_loss.item():.4f}") - - diff --git a/data/train_tokenizer.py b/data/train_tokenizer.py deleted file mode 100644 index fc98c47..0000000 --- a/data/train_tokenizer.py +++ /dev/null @@ -1,43 +0,0 @@ -import os -from tokenizers import ByteLevelBPETokenizer - -tokenizer = ByteLevelBPETokenizer() - -# Learn a vocab from your hand histories. -# vocab_size is small on purpose -- your domain vocabulary is tiny -# compared to English, so we don't need GPT-2's 50k tokens. -tokenizer.train( - files=["data/hands.txt"], - vocab_size=4000, - min_frequency=2, - special_tokens=["<|startoftext|>", "<|pad|>"], -) - -# Create tokenizer directory if it doesn't exist -os.makedirs("tokenizer", exist_ok=True) -tokenizer.save("tokenizer/tokenizer.json") # saves in the correct JSON format - -# Wrap it in a Transformers-compatible tokenizer class so it works with -# AutoModel / TRL later. -from transformers import PreTrainedTokenizerFast - -hf_tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer/tokenizer.json") -hf_tokenizer.add_special_tokens({ - "pad_token": "<|pad|>", - "bos_token": "<|startoftext|>", - "eos_token": "<|startoftext|>", -}) -hf_tokenizer.save_pretrained("tokenizer") - -# Sanity check: tokenize one real hand and print it. -sample = open("data/hands.txt", encoding="utf-8").readline().strip() -ids = hf_tokenizer(sample)["input_ids"] -print("Sample hand:", sample) -print("Token count:", len(ids)) - -# Check the length distribution across all hands -- this tells us -# what context length (max_position_embeddings) the model needs. -lines = open("data/hands.txt", encoding="utf-8").readlines() -lengths = [len(hf_tokenizer(l.strip())["input_ids"]) for l in lines] -print("Max tokens in a hand:", max(lengths)) -print("Avg tokens per hand:", sum(lengths) / len(lengths)) diff --git a/docs/EXPERIMENTS.md b/docs/EXPERIMENTS.md new file mode 100644 index 0000000..28463e7 --- /dev/null +++ b/docs/EXPERIMENTS.md @@ -0,0 +1,555 @@ +# W&B Experiment Catalog — Poker AI + +This document is a complete catalog of experiments you can run against this repository’s models and configurations. When Weights & Biases is configured, runs log to the shared project **[`mooslin-university-of-wisconsin-madison/poker-ai`](https://wandb.ai/mooslin-university-of-wisconsin-madison/poker-ai)**. Use it to compare baselines, ablations, and training backends with a consistent evaluation story. + +**W&B is optional.** Trainers only call `wandb.init` / set `report_to="wandb"` when an API key is present, `WANDB_MODE` is `offline`/`online`/`shared`, or you are already logged in. Otherwise training proceeds with **no** W&B logging (`report_to="none"`). Catalog and sweep workflows that need comparable panels should enable W&B explicitly (see below) or pass `--require-wandb` to the launcher. + +| Script | Run name (default) | Key metrics (when W&B is on) | +|--------|--------------------|------------------------------| +| `scripts/train_bigram.py` | `bigram-baseline` (or `WANDB_NAME`) | `train/loss`, `val/loss`, `final_val_loss`, `random_baseline_loss` | +| `scripts/train_gpt2.py` | `gpt2-raw` (or `WANDB_NAME`) | `train/loss`, `val/loss`, `final_val_loss` | +| `scripts/train_trl.py` | `gpt2-trl` (or `WANDB_NAME`) | HF Trainer / TRL metrics (`loss`, `eval_loss`, …) via `report_to="wandb"` when configured | + +--- + +## Shared setup (run once before any experiment) + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -e ".[dev]" + +python scripts/prepare_data.py # -> data/hands.txt, data/hands_clean.txt +python scripts/train_tokenizer.py # -> artifacts/tokenizer/ +``` + +**Enable W&B for catalog / dashboard runs** (pick one): + +```bash +wandb login # online logging +# or: +export WANDB_MODE=offline # log locally, sync later +# force off (no logging, no prompts): +export WANDB_MODE=disabled +``` + +**Offline compute (e.g. CHTC):** + +```bash +export WANDB_MODE=offline +# … run training … +wandb sync wandb/offline-run-* +``` + +**W&B hygiene (recommended whenever you want comparable panels):** + +```bash +export WANDB_ENTITY=mooslin-university-of-wisconsin-madison +export WANDB_PROJECT=poker-ai +export WANDB_RUN_GROUP="" # e.g. model-comparison +export WANDB_TAGS="exp01,gpt2,baseline" +# optionally: export WANDB_NAME="gpt2-raw-lr3e4" +``` + +The launcher prints whether W&B is active. For sweeps or must-log experiments, fail loudly if it is not: + +```bash +python scripts/run_experiment.py gpt2 --require-wandb --group capacity --tags exp-b1 +``` + +--- + +## What “good” looks like (shared evaluation rubric) + +Primary metric across runs: **`final_val_loss`** (or TRL’s epoch `eval_loss`) on the **action-only** (completion-masked) objective. Lower is better. + +| Reference | Expected ballpark | Notes | +|-----------|-------------------|-------| +| Uniform random over vocab | `ln(V) ≈ ln(4000) ≈ 8.3` | Logged by bigram as `random_baseline_loss` | +| Bigram (default) | **~1.53** val loss | Repo baseline; if GPT-2 does not beat this, upstream is broken | +| Default GPT-2 (6L/256/8H, 3 epochs) | **Clearly below bigram**, typically well under ~1.3 if training is healthy | Exact number depends on data size / split | +| GPT-2 raw vs TRL | Should be **roughly comparable** at matched hparams | Large gaps imply masking or length mismatch | + +Always also check: + +1. **Train vs val gap** — large gap ⇒ overfitting or noisy small eval set. +2. **Action-type distribution** — data is typically FOLD-heavy; raw CE loss can look good while RAISE/BET sizing is weak. +3. **Smoke predictions** — `python scripts/predict.py "," --model artifacts/models/...` should emit plausible actions (`FOLD`, `CALL`, `RAISE …BB`, …), not garbage tokens. +4. **Truncation** — after tokenizer training, if max hand length ≫ `n_positions` (384), long contexts are left-truncated; architecture/context experiments matter more in that regime. + +--- + +## How to override hyperparameters + +Use the experiment launcher (preferred): + +```bash +python scripts/run_experiment.py bigram --group model-comparison --tags exp-a1,baseline +python scripts/run_experiment.py gpt2 --n-layer 4 --lr 1e-4 --name gpt2-depth4 --group depth-sweep --tags exp-b1 +python scripts/run_experiment.py trl --epochs 5 --batch-size 16 --group optim --tags exp-c3 +python scripts/run_experiment.py majority --group imbalance --tags exp-f2 +python scripts/run_experiment.py features --method rf --group alt-approaches --tags exp-i2 +python scripts/run_experiment.py weighted-gpt2 --group imbalance --tags exp-f3 +python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256 +``` + +Or construct `GPT2Hyperparams` in Python / edit `src/pokerai/config.py`. For tokenizer vocab ablations, retrain with: + +```python +from pokerai.data.tokenizer import train_tokenizer +train_tokenizer(vocab_size=2000) # then re-run model training +``` + +Tag every W&B run with the **exact** config so panels stay comparable. `--name` / `--group` / `--tags` set `WANDB_NAME`, `WANDB_RUN_GROUP`, and `WANDB_TAGS`. Without W&B configured, those flags are still accepted but nothing is logged unless you pass `--require-wandb` (which exits with an error). + +--- + +# Experiment families + +## Family A — Model comparison (must-run showcase) + +These runs are the headline W&B dashboard: same data, same tokenizer, same eval definition. + +### EXP-A1 — Bigram action-only baseline + +| | | +|--|--| +| **Config** | `BIGRAM_LR=1e-2`, `BIGRAM_BATCH_SIZE=4096`, `BIGRAM_STEPS=2000`; action-token pairs only (`masked_action_only=True`) | +| **Command** | `python scripts/run_experiment.py bigram --group model-comparison --name bigram-baseline --tags exp-a1` | +| **Showcases** | Strong statistical baseline that only models P(next action token \| previous token). Fair comparison vs GPT-2 because both use action-region supervision. | +| **Look for** | `final_val_loss ≈ 1.53`; clearly below `random_baseline_loss` (~8.3). Flat late-curve ⇒ converged. | + +### EXP-A2 — GPT-2 raw loop (default architecture) + +| | | +|--|--| +| **Config** | `n_positions=384`, `n_embd=256`, `n_layer=6`, `n_head=8`, `lr=3e-4`, `batch=32`, `epochs=3`, AdamW; action-only CE via `encode_with_mask` | +| **Command** | `python scripts/run_experiment.py gpt2 --group model-comparison --name gpt2-raw --tags exp-a2` | +| **Showcases** | Whether a small randomly-initialized decoder beats bigram when it can condition on full game state. | +| **Look for** | `final_val_loss` **≪** bigram (~1.53). Smooth `train/loss` decline; end-of-epoch `val/loss` improving then stabilizing. If val ≥ bigram, inspect tokenizer, comma split, or masking. | + +### EXP-A3 — GPT-2 via TRL SFT (`completion_only_loss=True`) + +| | | +|--|--| +| **Config** | Same `GPT2Hyperparams` as A2; `SFTConfig(completion_only_loss=True, max_length=n_positions, eval_strategy="epoch", fp16=cuda)` | +| **Command** | `python scripts/run_experiment.py trl --group model-comparison --name gpt2-trl --tags exp-a3` | +| **Showcases** | Equivalence (or gap) between the hand-rolled loop and HF/TRL’s completion-only path — critical for trusting production training. | +| **Look for** | `eval_loss` within a small band of A2’s `final_val_loss` (same seed/data). Materially worse TRL ⇒ prompt/completion split or `max_length` mismatch. Materially better + fp16 ⇒ note precision/speed tradeoff. | + +### EXP-A4 — Side-by-side W&B panel (A1–A3) + +| | | +|--|--| +| **Config** | Identical data + tokenizer; group `WANDB_RUN_GROUP=model-comparison` | +| **How to test** | Run A1→A3; in W&B create a parallel coordinates / line plot of `val/loss` (or `eval_loss`) and a bar of `final_val_loss`. | +| **Showcases** | The repo’s core claim: **GPT-2 > bigram**, and **raw ≈ TRL**. | +| **Look for** | Ordering: random ≫ bigram > GPT-2 (raw ≈ TRL). No GPT-2 run should lose to bigram after 3 epochs on the default stack. | + +--- + +## Family B — Architecture capacity + +Hold training recipe fixed (`lr=3e-4`, `batch=32`, `epochs=3`, `n_positions=384`) unless noted. Prefer `train_gpt2.py` or `train_trl.py` consistently within a sweep. + +### EXP-B1 — Depth sweep (`n_layer`) + +| | | +|--|--| +| **Configs** | `n_layer ∈ {2, 4, 6 (default), 8, 12}` with `n_embd=256`, `n_head=8` | +| **How to test** | `python scripts/run_experiment.py gpt2 --n-layer L --name gpt2-depth-L --group depth-sweep --tags exp-b1,depth-L` for each L. | +| **Showcases** | Returns to depth for long-horizon table state → action mapping. | +| **Look for** | Val loss decreasing then plateauing; diminishing returns past 6–8 layers. Rising val + falling train ⇒ overfit / too much capacity for data size. | + +### EXP-B2 — Width sweep (`n_embd`) + +| | | +|--|--| +| **Configs** | `n_embd ∈ {128, 256 (default), 384, 512}`; keep `n_head` dividing `n_embd` (e.g. 4/8/8/8 or 8/8/12/16) | +| **How to test** | Pair each width with a valid `n_head`; log `n_params` (already in raw GPT-2 W&B config). | +| **Showcases** | Embedding/capacity vs compute; useful for choosing a deployable size. | +| **Look for** | Pareto curve: val loss vs parameter count. 128d should underfit relative to 256d; 512d should help only if data is rich enough. | + +### EXP-B3 — Attention head count (`n_head`) + +| | | +|--|--| +| **Configs** | Fixed `n_embd=256`, `n_head ∈ {4, 8 (default), 16}` (`n_embd % n_head == 0`) | +| **How to test** | Three runs; same seed/data. | +| **Showcases** | Whether multi-head diversity matters for poker state features (position, stacks, board texture proxies in text). | +| **Look for** | Usually mild effect vs depth/width. Large degradation at extreme head sizes ⇒ unstable training or too-thin heads. | + +### EXP-B4 — Context length (`n_positions`) + +| | | +|--|--| +| **Configs** | `n_positions ∈ {128, 256, 384 (default), 512}`; must match tokenizer `model_max_length` and TRL `max_length` | +| **How to test** | After choosing length L, set `GPT2Hyperparams(n_positions=L)` and ensure encode/TRL max length = L. Compare fraction of truncated hands (from tokenizer length stats). | +| **Showcases** | Cost of left-truncation on long multi-street histories vs memory/speed. | +| **Look for** | If many hands exceed 128–256 tokens, raising context should **materially** cut val loss. If almost all hands fit in 256, 384→512 yields little gain. | + +### EXP-B5 — Tiny vs default vs “large-small” GPT-2 + +| | | +|--|--| +| **Configs** | **Tiny:** 2L / 128d / 4H; **Default:** 6L / 256d / 8H; **Large-small:** 8L / 512d / 8H | +| **How to test** | Three named runs; optionally match wall-clock by adjusting epochs. | +| **Showcases** | Compact “poster” comparison of capacity classes for demos. | +| **Look for** | Clear ordering Tiny > Default ≥ Large-small on val loss (loss numbers: Tiny highest). Large-small only wins if val improves without diverging. | + +--- + +## Family C — Optimization & schedule + +Default architecture unless noted. + +### EXP-C1 — Learning-rate sweep + +| | | +|--|--| +| **Configs** | `lr ∈ {1e-4, 3e-4 (default), 1e-3, 3e-3}` | +| **How to test** | Four GPT-2 runs (raw or TRL); identical other hparams. | +| **Showcases** | Stability of AdamW on randomly-initialized GPT-2 for this domain. | +| **Look for** | `3e-4` near-optimal. `1e-4` slower but stable. `≥1e-3` may spike train loss / worse val. Prefer lowest stable `final_val_loss`. | + +### EXP-C2 — Batch size sweep + +| | | +|--|--| +| **Configs** | `batch_size ∈ {8, 16, 32 (default), 64}` (fit GPU memory; TRL uses `per_device_*_batch_size`) | +| **How to test** | Keep `lr` fixed first; optional follow-up with linear LR scaling. | +| **Showcases** | Gradient noise vs throughput; sensitivity of small poker LM to batch statistics. | +| **Look for** | Very small batches: noisier curves, possibly better generalization. Very large: faster steps but possible val degradation without LR retune. | + +### EXP-C3 — Epoch / compute budget + +| | | +|--|--| +| **Configs** | `num_epochs ∈ {1, 3 (default), 5, 10}` | +| **How to test** | Log val every epoch; compare best-epoch vs final. | +| **Showcases** | Underfitting vs overfitting horizon on the 95/5 split. | +| **Look for** | Val improving through epoch 3; later epochs: either small gains or rising val (overfit). Report **best** val, not only last. | + +### EXP-C4 — Bigram optimization sanity + +| | | +|--|--| +| **Configs** | `BIGRAM_STEPS ∈ {500, 2000 (default), 8000}`; `BIGRAM_LR ∈ {1e-3, 1e-2, 5e-2}` | +| **How to test** | `train_bigram.main(n_steps=..., lr=...)` | +| **Showcases** | That ~1.53 is a converged baseline, not an undertrained artifact. | +| **Look for** | Default 2k steps ≈ converged (extra steps ≈ flat). Much worse final val at weird LRs ⇒ don’t trust GPT-2 comparisons until bigram is retuned. | + +--- + +## Family D — Tokenizer & representation + +Retrain tokenizer **and** all models when vocab/specials change (token IDs shift). + +### EXP-D1 — Vocabulary size + +| | | +|--|--| +| **Configs** | `VOCAB_SIZE ∈ {1000, 2000, 4000 (default), 8000}` | +| **How to test** | For each V: `train_tokenizer(vocab_size=V)` → bigram + GPT-2 default. Compare val loss **and** avg tokens/hand. | +| **Showcases** | BPE granularity tradeoff: smaller vocab ⇒ longer sequences / more truncation; larger ⇒ rarer tokens, harder action modeling. | +| **Look for** | Sweet spot near 4k. Collapse in rare RAISE sizing tokens at tiny V; little gain + slower train at 8k. | + +### EXP-D2 — Train tokenizer on raw vs cleaned text + +| | | +|--|--| +| **Configs** | Tokenizer on `hands.txt` (default) vs `hands_clean.txt` | +| **How to test** | Point `train_tokenizer(hands_path=...)` at each; train identical GPT-2. | +| **Showcases** | Impact of normalizing `CALL 0BB` / `FOLD0BB` on the subword inventory. | +| **Look for** | Cleaned tokenizer should not hurt and may slightly help action-token consistency. Large gap ⇒ noisy special-case strings in vocab. | + +### EXP-D3 — Distinct BOS/EOS (regression guard) + +| | | +|--|--| +| **Configs** | Correct: distinct `<\|startoftext\|>` / `<\|endoftext\|>` (repo default). Ablation only if you intentionally break this. | +| **How to test** | Unit test `tests/test_config.py::test_special_tokens_are_distinct`; optionally train a broken tokenizer where BOS=EOS and compare generation stopping. | +| **Showcases** | Why the README insists BOS ≠ EOS for boundary learning / `generate` stopping. | +| **Look for** | Broken specials ⇒ worse val and/or runaway or empty generations in `predict.py`. | + +--- + +## Family E — Data & supervision + +### EXP-A reminder — always use the same split seed + +Default `TEST_SIZE=0.05`, `SPLIT_SEED=42` in `load_text_split`. Changing seed without retagging makes W&B comparisons invalid. + +### EXP-E1 — Train-set size scaling + +| | | +|--|--| +| **Configs** | Subsample train to `{10%, 25%, 50%, 100%}` of `split["train"]` (fixed test set). | +| **How to test** | Filter the HF `Dataset` before encoding / TRL map; keep eval identical. | +| **Showcases** | Data efficiency of GPT-2 vs bigram; whether more hands still buy loss. | +| **Look for** | Monotone val improvement with more data. GPT-2 gap vs bigram widening with scale. Plateau ⇒ architecture/label noise limited, not data-limited. | + +### EXP-E2 — Cleaning on vs off + +| | | +|--|--| +| **Configs** | Train on `hands_clean.txt` (default) vs raw `hands.txt` | +| **How to test** | Pass `hands_path=` into `train_*` mains; same tokenizer (note D2 interaction). | +| **Showcases** | Value of the repo’s `CALL 0BB` / `FOLD…BB` normalization. | +| **Look for** | Cleaned data ≤ raw val loss; fewer pathological predicted actions like `CALL 0BB`. | + +### EXP-E3 — Official HF split vs random 95/5 re-split + +| | | +|--|--| +| **Configs** | **A:** `prepare_data.py` + `load_text_split` (random 5%). **B:** `scripts/convert_poker_dataset.py` train/test files (dataset’s own split). | +| **How to test** | Both paths write **comma-separated** `context,action` lines (same as `to_line` / `split_prompt_completion`). For **B**, point trainers at `data/hands_train.txt` / `data/hands_test.txt` (or load those files instead of `load_text_split` on `hands_clean.txt`). Convert also writes `hands_clean.txt` for the default pipeline. | +| **Showcases** | Generalization under the dataset author’s held-out split vs an i.i.d. reshuffle (possible leakage if correlated hands). | +| **Look for** | Val loss higher on the official test split is common and more honest. Large train/test distribution shift ⇒ report both. | + +### EXP-E4 — Completion-only vs full-sequence loss (ablation) + +| | | +|--|--| +| **Configs** | TRL: `completion_only_loss=True` (default) vs `False`. Raw loop: mask action-only vs supervise all tokens. | +| **How to test** | Toggle TRL flag; for raw loop temporarily label all non-pad tokens. **Compare action-region CE only** at eval time for fairness. | +| **Showcases** | Core design claim: masking state tokens focuses capacity on decisions. | +| **Look for** | Full-sequence training may lower *token* loss (easy state copying) but **action-only eval** should favor completion-only. If not, masking implementation is suspect. | + +### EXP-E5 — Left-truncation stress test + +| | | +|--|--| +| **Configs** | Artificially low `n_positions` (e.g. 64 or 128) vs 384 on the same data. | +| **How to test** | Pair with B4; measure % of examples with `len(ids)==max_length` after encode. | +| **Showcases** | How much poker context the model actually needs before the decision point. | +| **Look for** | Sharp val degradation when truncation eats stacks/board/prior actions; confirms front-drop policy keeps the action (labels still present) but drops useful state. | + +--- + +## Family F — Class imbalance & decision quality + +Raw CE is insufficient alone. These experiments make W&B charts **action-aware** (FOLD / CALL / RAISE / CHECK / BET). + +Post-train GPT-2 eval and `scripts/evaluate.py` write confusion matrices, per-class F1 bars, and occlusion feature-importance charts under `/report/` and log them to W&B when configured. + +### EXP-F1 — Per-action-type eval metrics + +| | | +|--|--| +| **Configs** | Default GPT-2 (or any saved HF checkpoint) | +| **How to test** | Automatic after `run_experiment.py gpt2` / `weighted-gpt2`, or: `python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256`. Metrics: `eval/action/{fold,call,raise,...}/{precision,recall,f1}` + confusion plot. | +| **Showcases** | That low overall loss can hide failure on minority aggressive actions. | +| **Look for** | High FOLD accuracy, weaker RAISE/BET. Prefer models that improve minority classes without collapsing to always-FOLD. | + +### EXP-F2 — Majority-class baseline + +| | | +|--|--| +| **Configs** | Constant predictor = most frequent train action type (usually FOLD). | +| **How to test** | `python scripts/run_experiment.py majority --group imbalance --tags exp-f2` | +| **Showcases** | Floor for classification-style metrics (complementary to bigram’s token CE). | +| **Look for** | Neural / feature models should beat majority accuracy on non-FOLD slices even if overall accuracy looks “high” from FOLD dominance. | + +### EXP-F3 — Weighted GPT-2 loss on action types + +| | | +|--|--| +| **Configs** | Inverse-frequency weights on examples by action type (`class_weight=True`). | +| **How to test** | `python scripts/run_experiment.py weighted-gpt2 --group imbalance --tags exp-f3` | +| **Showcases** | Whether rebalancing improves RAISE/BET quality at a tolerable cost to FOLD CE. | +| **Look for** | Better minority `eval/action/raise/f1` (and BET); mild regression in overall `val/loss` is OK if decision quality improves. | + +--- + +## Family G — Training backend & systems + +### EXP-G1 — Precision: fp16 vs fp32 (TRL) + +| | | +|--|--| +| **Configs** | TRL `fp16=True` (default on CUDA) vs `fp16=False` | +| **How to test** | Two `train_trl` runs; same hparams. | +| **Showcases** | Numerical sensitivity of this small GPT-2. | +| **Look for** | Nearly matching `eval_loss`; fp16 faster / less memory. Large divergence ⇒ stick to fp32 for reporting. | + +### EXP-G2 — Device parity (CUDA vs CPU/MPS) + +| | | +|--|--| +| **Configs** | Default hparams; force device via environment / availability. | +| **How to test** | Short 1-epoch smoke on CPU vs full GPU run; compare loss trajectories at matched step counts. | +| **Showcases** | Reproducibility across hardware for contributors. | +| **Look for** | Same qualitative curves; absolute equality not required. | + +### EXP-G3 — Seed reproducibility + +| | | +|--|--| +| **Configs** | 3 seeds for default GPT-2 (set `torch.manual_seed`, `numpy`, `random`, and DataLoader generators). | +| **How to test** | Report mean ± std of `final_val_loss`. | +| **Showcases** | Run-to-run variance for claiming “GPT-2 beats bigram.” | +| **Look for** | Tight cluster; GPT-2 mean − bigram gap ≫ seed std. | + +### EXP-G4 — Offline W&B sync path + +| | | +|--|--| +| **Configs** | `WANDB_MODE=offline` then `wandb sync` | +| **How to test** | One bigram or 1-epoch GPT-2 on a node without egress; sync later. | +| **Showcases** | Cluster-friendly tracking workflow documented in README. | +| **Look for** | Identical charts after sync; no missing summary fields (`final_val_loss`). | + +--- + +## Family H — Inference / qualitative showcase + +### EXP-H1 — Greedy prediction gallery + +| | | +|--|--| +| **Configs** | Best GPT-2 raw vs TRL checkpoints; `predict_action(..., max_new_tokens=16)`, `do_sample=False` | +| **How to test** | Curate ~20 held-out states spanning FOLD/CALL/RAISE/BET/CHECK; log a W&B Table of `{state, gold, pred_raw, pred_trl}`. | +| **Showcases** | Human-readable differences that loss alone cannot show (sizing format, illegal actions). | +| **Look for** | Valid action syntax; agreement between raw/TRL; failures concentrated on rare sizes / multi-token raises. | + +### EXP-H2 — Sampling vs greedy + +| | | +|--|--| +| **Configs** | Temporarily enable `do_sample=True` with low temperature vs greedy (code tweak in `predict.py`). | +| **How to test** | Same prompts; compare diversity and validity rates. | +| **Showcases** | Whether the LM assigns a peaked action distribution or a diffuse one. | +| **Look for** | Greedy preferred for decision agents; sampling should not invent malformed suffixes. | + +--- + +## Family I — Alternative modeling approaches + +Different paradigms from the GPT-2 LM — useful for feature importance, imbalance floors, and comparing text LMs to tabular classifiers. + +### EXP-I1 — Majority action-type baseline (see also F2) + +| | | +|--|--| +| **Command** | `python scripts/run_experiment.py majority --group alt-approaches --tags exp-i1` | +| **Showcases** | Non-neural floor; charts include confusion + per-class F1. | +| **Look for** | Near-100% FOLD recall, ~0 on RAISE/BET — the imbalance story in one plot. | + +### EXP-I2 — RandomForest on structured hand features + +| | | +|--|--| +| **Command** | `python scripts/run_experiment.py features --method rf --group alt-approaches --tags exp-i2` | +| **Showcases** | Tabular ML with native `feature_importances_` (pot, stacks, street, aggression counts, hole cards, …). Logs importance bar chart + action confusion. | +| **Look for** | Which engineered features drive CALL vs FOLD; whether RF beats majority on RAISE/BET F1. | + +### EXP-I3 — Balanced LogisticRegression on the same features + +| | | +|--|--| +| **Command** | `python scripts/run_experiment.py features --method logreg --group alt-approaches --tags exp-i3` | +| **Showcases** | Linear, class-balanced alternative; importance = mean \|coefficient\|. | +| **Look for** | Comparable story to RF with a simpler decision boundary; good sanity check on feature scale. | + +### EXP-I4 — Class-weighted GPT-2 (see also F3) + +| | | +|--|--| +| **Command** | `python scripts/run_experiment.py weighted-gpt2 --group alt-approaches --tags exp-i4` | +| **Showcases** | Same architecture as A2, different training objective (inverse-frequency example weights) + full action eval charts. | +| **Look for** | Lift on minority action F1 vs unweighted `gpt2` at matched hparams. | + +### EXP-I5 — Occlusion feature importance on a trained LM + +| | | +|--|--| +| **Command** | `python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 128 --max-importance 64` | +| **Showcases** | Ablate `[TABLE_CONFIGURATION]`, stacks, pot, streets, hole cards; importance = Δ action accuracy. | +| **Look for** | Large drops when masking stacks / aggression history; confirms the model uses poker state, not just priors. | + +--- + +## Family J — Suggested W&B sweeps (automation) + +Minimal sweep for a compelling public project page: + +```yaml +# docs/wandb_sweep_model_compare.yaml +program: scripts/run_experiment.py +entity: mooslin-university-of-wisconsin-madison +project: poker-ai +method: grid +parameters: + # express via a runner that reads WANDB/sweep env into GPT2Hyperparams + model_kind: + values: ["bigram", "gpt2-raw", "gpt2-trl"] +``` + +Capacity sweep (raw GPT-2): + +| Parameter | Values | +|-----------|--------| +| `n_layer` | 2, 4, 6, 8 | +| `n_embd` | 128, 256, 512 | +| `learning_rate` | 1e-4, 3e-4, 1e-3 | +| `num_epochs` | 3 | + +**Constraint:** `n_embd % n_head == 0` (fix `n_head=8` when possible). + +--- + +## Recommended execution order (showcase path) + +Run in this order for a clean W&B narrative: + +1. **A1** Bigram baseline +2. **A2** GPT-2 raw default +3. **A3** GPT-2 TRL default +4. **A4** Dashboard: model comparison +5. **C1** LR sweep on the winning backend +6. **B1 + B2** Depth/width on the winning LR +7. **B4** Context length (guided by tokenizer length histogram) +8. **E1** Data scaling +9. **F1 + I2 + I5** Per-action metrics, RF feature importance, occlusion charts (demo-ready) +10. **F2 + F3 / I4** Majority floor vs weighted GPT-2 + +--- + +## Scoring card (paste into W&B notes) + +For each completed experiment, record: + +| Field | Value | +|-------|-------| +| Run name / group / tags | | +| Trainer (`bigram` / `gpt2-raw` / `gpt2-trl` / `majority` / `features` / `weighted-gpt2`) | | +| Architecture (`n_layer/n_embd/n_head/n_positions`) | | +| Optim (`lr`, `batch`, `epochs` or `steps`) | | +| Data (`hands_clean` path, split seed, subsample %) | | +| Tokenizer vocab / max tokens observed | | +| `final_val_loss` or best `eval_loss` | | +| Action metrics (`accuracy`, `macro_f1`, per-class F1) | | +| vs bigram Δ / vs majority Δ | | +| Notes (truncation %, qualitative fails, top features) | | + +**Pass criteria for the default stack:** GPT-2 (raw and TRL) both beat bigram `final_val_loss` by a clear margin; raw≈TRL; predictions are syntactically valid poker actions on a smoke gallery; action-type charts show non-zero RAISE/BET recall (not pure majority-FOLD). + +--- + +## Mapping to repo knobs + +| Knob | Location | +|------|----------| +| GPT-2 architecture & optim | `src/pokerai/config.py` → `GPT2Hyperparams` | +| Bigram optim | `BIGRAM_LR`, `BIGRAM_BATCH_SIZE`, `BIGRAM_STEPS` | +| W&B project | `WANDB_ENTITY` / `WANDB_PROJECT` → `mooslin-university-of-wisconsin-madison/poker-ai` (logging only when W&B is configured; see setup) | +| Masked encode | `pokerai.data.encode_with_mask` (offset-based boundary; keep-end truncation) | +| TRL completion-only | Pre-tokenized via `encode_with_mask` + `skip_prepare_dataset=True` (action labels already `-100`-masked) | +| Paths | `HANDS_CLEAN`, `TOKENIZER_DIR`, `MODEL_*_DIR` | +| Hand format | Always `context,action` (last comma). Converter and prepare share `to_line`. | +| Action eval / charts | `pokerai.eval` + `scripts/evaluate.py` | +| Structured features | `pokerai.features` → RF / LogReg trainers | +| Class-weighted GPT-2 | `scripts/run_experiment.py weighted-gpt2` | + +Experiments marked as needing further instrumentation (**H2** sampling toggle) remain optional polish. **F1–F3** and **I1–I5** are implemented via the CLI trainers and `evaluate.py`. diff --git a/docs/experiments-v2.md b/docs/experiments-v2.md new file mode 100644 index 0000000..4404922 --- /dev/null +++ b/docs/experiments-v2.md @@ -0,0 +1,9 @@ +## Python Scripts + +```python +python scripts/run_experiment.py majority # imbalance floor +python scripts/run_experiment.py features --method rf # RF + feature importances +python scripts/run_experiment.py features --method logreg +python scripts/run_experiment.py weighted-gpt2 # class-weighted GPT-2 +python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256 +``` \ No newline at end of file diff --git a/docs/wandb_sweep_gpt2_capacity.yaml b/docs/wandb_sweep_gpt2_capacity.yaml new file mode 100644 index 0000000..21e232a --- /dev/null +++ b/docs/wandb_sweep_gpt2_capacity.yaml @@ -0,0 +1,57 @@ +# Architecture + LR grid for raw GPT-2 (Family B + C1). +# Use with scripts/run_experiment.py via a wandb agent entrypoint, or launch +# individual points manually, e.g.: +# +# python scripts/run_experiment.py gpt2 --require-wandb --n-layer 4 --n-embd 256 \ +# --lr 3e-4 --group capacity --name gpt2-l4-e256-lr3e4 --tags sweep,capacity +# +# W&B is optional for ordinary training; sweeps should use --require-wandb so a +# missing login fails loudly instead of training with silent no-op logging. +# +# wandb sweep docs/wandb_sweep_gpt2_capacity.yaml +# wandb agent + +program: scripts/run_experiment.py +entity: mooslin-university-of-wisconsin-madison +project: poker-ai +method: grid +metric: + name: final_val_loss + goal: minimize +command: + - ${env} + - python + - scripts/run_experiment.py + - gpt2 + - --require-wandb + - --n-layer + - ${args.n_layer} + - --n-embd + - ${args.n_embd} + - --n-head + - ${args.n_head} + - --n-positions + - ${args.n_positions} + - --lr + - ${args.learning_rate} + - --batch-size + - ${args.batch_size} + - --epochs + - ${args.num_epochs} + - --group + - capacity-sweep +parameters: + n_layer: + values: [2, 4, 6, 8] + n_embd: + values: [128, 256, 512] + n_head: + values: [8] + n_positions: + values: [384] + learning_rate: + values: [0.0001, 0.0003, 0.001] + batch_size: + values: [32] + num_epochs: + values: [3] diff --git a/docs/wandb_sweep_model_compare.yaml b/docs/wandb_sweep_model_compare.yaml new file mode 100644 index 0000000..66fc727 --- /dev/null +++ b/docs/wandb_sweep_model_compare.yaml @@ -0,0 +1,35 @@ +# Headline model comparison — launch the three trainers in one W&B group. +# Prefer the documented manual commands in docs/EXPERIMENTS.md Family A, or: +# +# python scripts/run_experiment.py bigram --require-wandb --group model-comparison --tags exp-a1 +# python scripts/run_experiment.py gpt2 --require-wandb --group model-comparison --tags exp-a2 +# python scripts/run_experiment.py trl --require-wandb --group model-comparison --tags exp-a3 +# +# W&B is optional for ordinary training, but sweeps need a configured project +# (wandb login or WANDB_MODE=offline). Use --require-wandb so a missing login +# fails loudly instead of training with silent no-op logging. +# +# Sweep automation (requires wandb agent machine with data + tokenizer ready): +# wandb sweep docs/wandb_sweep_model_compare.yaml +# wandb agent + +program: scripts/run_experiment.py +entity: mooslin-university-of-wisconsin-madison +project: poker-ai +method: grid +metric: + name: final_val_loss + goal: minimize +command: + - ${env} + - python + - scripts/run_experiment.py + - ${args.trainer} + - --require-wandb + - --group + - model-comparison + - --tags + - model-compare +parameters: + trainer: + values: ["bigram", "gpt2", "trl"] diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..bf5f072 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,44 @@ +[build-system] +requires = ["setuptools>=68", "wheel"] +build-backend = "setuptools.build_meta" + +[project] +name = "pokerai" +version = "0.2.0" +description = "Train a small GPT-2 to predict poker actions from game state" +readme = "README.md" +requires-python = ">=3.10" +dependencies = [ + "torch>=2.0.0", + "transformers>=4.40.0", + "datasets>=2.0.0", + "tokenizers>=0.15.0", + "trl>=0.14.0", + "wandb>=0.16.0", + "pyarrow>=14.0.0", + "accelerate>=0.26.0", + "scikit-learn>=1.3.0", + "matplotlib>=3.7.0", + "joblib>=1.3.0", +] + +[project.optional-dependencies] +dev = ["pytest>=7.0.0"] + +[project.scripts] +pokerai-prepare = "pokerai.data.prepare:main" +pokerai-tokenizer = "pokerai.data.tokenizer:main" +pokerai-bigram = "pokerai.training.train_bigram:main" +pokerai-gpt2 = "pokerai.training.train_gpt2:main" +pokerai-trl = "pokerai.training.train_trl:main" +pokerai-majority = "pokerai.training.train_majority:main" +pokerai-features = "pokerai.training.train_features:main" +pokerai-weighted-gpt2 = "pokerai.training.train_weighted_gpt2:main" +pokerai-predict = "pokerai.inference:main" + +[tool.setuptools.packages.find] +where = ["src"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +pythonpath = ["src"] diff --git a/pyrightconfig.json b/pyrightconfig.json new file mode 100644 index 0000000..926c6e0 --- /dev/null +++ b/pyrightconfig.json @@ -0,0 +1,8 @@ +{ + "venvPath": ".", + "venv": ".venv", + "pythonVersion": "3.11", + "include": ["src", "scripts", "tests"], + "exclude": ["venv", ".venv", "**/__pycache__", "artifacts", "wandb"], + "extraPaths": ["src"] +} diff --git a/requirements.txt b/requirements.txt index 1be206f..3d0b2e2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,12 @@ torch>=2.0.0 -transformers>=5.0.0 +transformers>=4.40.0 datasets>=2.0.0 tokenizers>=0.15.0 -trl>=0.7.0 +trl>=0.14.0 +wandb>=0.16.0 +pyarrow>=14.0.0 +accelerate>=0.26.0 +scikit-learn>=1.3.0 +matplotlib>=3.7.0 +joblib>=1.3.0 +pytest>=7.0.0 diff --git a/scripts/convert_poker_dataset.py b/scripts/convert_poker_dataset.py new file mode 100644 index 0000000..3a7eb1e --- /dev/null +++ b/scripts/convert_poker_dataset.py @@ -0,0 +1,73 @@ +""" +Convert SoelMgd/Poker_Dataset (HuggingFace) into plain-text hand files +compatible with the pokerai training pipeline. + +Output (comma-separated prompt,action — same format as scripts/prepare_data.py): + data/hands.txt -- ALL hands (train+test), raw joined lines + data/hands_clean.txt -- cleaned lines (what trainers load by default) + data/hands_train.txt -- HF train split, cleaned + data/hands_test.txt -- HF test split, cleaned + +Prefer ``python scripts/prepare_data.py`` for the standard path. This script +additionally preserves the dataset's official train/test split files. +""" + +from __future__ import annotations + +from collections import Counter +from pathlib import Path + +from datasets import load_dataset + +from pokerai.config import DATA_DIR, HANDS_CLEAN, HANDS_RAW, HF_DATASET +from pokerai.data import clean_line, to_line + + +def convert_split(dataset_split, path: Path) -> list[str]: + lines = [ + clean_line(to_line(example["question"], example["answer"])) + for example in dataset_split + ] + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"Wrote {len(lines):,} lines to {path}") + return lines + + +def main() -> None: + print(f"Downloading {HF_DATASET} ...") + ds = load_dataset(HF_DATASET) + + print("Train split:", ds["train"]) + print("Test split:", ds["test"]) + + train_lines = convert_split(ds["train"], DATA_DIR / "hands_train.txt") + test_lines = convert_split(ds["test"], DATA_DIR / "hands_test.txt") + + # Combined raw + cleaned files for tokenizer / default trainers. + raw_lines = [ + to_line(q, a) + for split in ds + for q, a in zip(ds[split]["question"], ds[split]["answer"]) + ] + HANDS_RAW.parent.mkdir(parents=True, exist_ok=True) + HANDS_RAW.write_text("\n".join(raw_lines) + "\n", encoding="utf-8") + print(f"Wrote {len(raw_lines):,} lines to {HANDS_RAW}") + + cleaned = [clean_line(line) for line in raw_lines] + HANDS_CLEAN.write_text("\n".join(cleaned) + "\n", encoding="utf-8") + print(f"Wrote {len(cleaned):,} lines to {HANDS_CLEAN}") + + action_types = Counter(a.split()[0] for a in ds["train"]["answer"]) + print("\nAction type distribution (train):") + for action, count in action_types.most_common(): + print(f" {action:10s} {count:6,} ({100 * count / len(ds['train']):.1f}%)") + + print( + f"\nDone. Trainers load {HANDS_CLEAN} by default " + f"({len(train_lines):,} train / {len(test_lines):,} test split files also written)." + ) + + +if __name__ == "__main__": + main() diff --git a/scripts/evaluate.py b/scripts/evaluate.py new file mode 100644 index 0000000..4621d24 --- /dev/null +++ b/scripts/evaluate.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +"""Evaluate a saved model: action metrics, charts, feature importance. + +Examples: + python scripts/evaluate.py --model artifacts/models/gpt2 --max-examples 256 + python scripts/evaluate.py --model artifacts/models/gpt2_trl --require-wandb +""" + +from __future__ import annotations + +import argparse +import os +from pathlib import Path + +from pokerai.config import HANDS_CLEAN, MODEL_GPT2_DIR, MODEL_TRL_DIR +from pokerai.data import load_text_split +from pokerai.eval.report import run_lm_evaluation_report +from pokerai.inference.predict import load_model +from pokerai.training import ensure_wandb_project, wandb_is_configured + + +def main() -> None: + p = argparse.ArgumentParser(description=__doc__) + p.add_argument( + "--model", + type=Path, + default=MODEL_TRL_DIR if MODEL_TRL_DIR.exists() else MODEL_GPT2_DIR, + help="Directory with saved HF model + tokenizer", + ) + p.add_argument("--hands", type=Path, default=HANDS_CLEAN) + p.add_argument("--max-examples", type=int, default=256) + p.add_argument("--max-importance", type=int, default=64) + p.add_argument( + "--report-dir", + type=Path, + default=None, + help="Where to write PNG/JSON (default: /report)", + ) + p.add_argument("--split", choices=["test", "train"], default="test") + p.add_argument("--require-wandb", action="store_true") + p.add_argument("--name", default=None, help="WANDB_NAME override") + p.add_argument("--group", default=None, help="WANDB_RUN_GROUP") + p.add_argument("--tags", default="", help="Comma-separated WANDB_TAGS") + args = p.parse_args() + + if args.group: + os.environ["WANDB_RUN_GROUP"] = args.group + if args.tags: + os.environ["WANDB_TAGS"] = args.tags + if args.name: + os.environ["WANDB_NAME"] = args.name + + use_wandb = wandb_is_configured() + if args.require_wandb and not use_wandb: + raise SystemExit("error: --require-wandb set but W&B is not configured") + if use_wandb: + import wandb + from pokerai.config import WANDB_ENTITY, WANDB_PROJECT + + ensure_wandb_project() + wandb.init( + entity=WANDB_ENTITY, + project=WANDB_PROJECT, + name=os.environ.get("WANDB_NAME", f"eval-{args.model.name}"), + job_type="eval", + config={"model_dir": str(args.model), "max_examples": args.max_examples}, + ) + + model, tokenizer = load_model(args.model) + split = load_text_split(args.hands) + lines = [ex["text"] for ex in split[args.split]] + report_dir = args.report_dir or (args.model / "report") + summary = run_lm_evaluation_report( + model, + tokenizer, + lines, + report_dir=report_dir, + max_examples=args.max_examples, + max_importance=args.max_importance, + log_wandb=use_wandb, + ) + m = summary["metrics"] + print(f"accuracy={m['accuracy']:.4f} macro_f1={m['macro_f1']:.4f} n={m['n']}") + for label, scores in m["per_class"].items(): + print( + f" {label:6s} P={scores['precision']:.3f} R={scores['recall']:.3f} " + f"F1={scores['f1']:.3f} support={scores['support']}" + ) + if summary.get("importance"): + top = sorted(summary["importance"].items(), key=lambda kv: -kv[1])[:8] + print("Occlusion importance:", ", ".join(f"{k}={v:.3f}" for k, v in top)) + print(f"Report written to {report_dir}") + + if use_wandb: + import wandb + + wandb.finish() + + +if __name__ == "__main__": + main() diff --git a/scripts/predict.py b/scripts/predict.py new file mode 100644 index 0000000..ca06413 --- /dev/null +++ b/scripts/predict.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Predict a poker action from game state.""" + +from pokerai.inference import main + +if __name__ == "__main__": + main() diff --git a/scripts/prepare_data.py b/scripts/prepare_data.py new file mode 100644 index 0000000..f96c8fc --- /dev/null +++ b/scripts/prepare_data.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Download and clean poker hand histories.""" + +from pokerai.data.prepare import main + +if __name__ == "__main__": + main() diff --git a/scripts/run_experiment.py b/scripts/run_experiment.py new file mode 100644 index 0000000..36aba75 --- /dev/null +++ b/scripts/run_experiment.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Launch a tagged experiment with explicit hyperparameters. + +Trainers: + bigram, gpt2, trl — original LM / baseline stack + majority — always-predict majority action type + features — LogReg / RandomForest on structured features + weighted-gpt2 — GPT-2 with inverse-frequency action weights + +W&B logging is optional unless ``--require-wandb``. After generative training, +action-type charts (confusion, per-class F1) and occlusion feature importance +are logged when evaluation runs (GPT-2 / weighted-gpt2 / evaluate.py). + +Examples: + python scripts/run_experiment.py majority --group imbalance --tags exp-f2 + python scripts/run_experiment.py features --method rf --group alt-approaches + python scripts/run_experiment.py weighted-gpt2 --group imbalance --tags exp-f3 + python scripts/run_experiment.py gpt2 --n-layer 4 --eval-max-examples 128 +""" + +from __future__ import annotations + +import argparse +import os +import sys +from pathlib import Path + +from pokerai.config import ( + BIGRAM_BATCH_SIZE, + BIGRAM_LR, + BIGRAM_STEPS, + GPT2_BATCH_SIZE, + GPT2_EPOCHS, + GPT2_LR, + GPT2Hyperparams, + HANDS_CLEAN, + N_EMBD, + N_HEAD, + N_LAYER, + N_POSITIONS, + WANDB_ENTITY, + WANDB_PROJECT, +) +from pokerai.training import ensure_wandb_project, wandb_is_configured + + +TRAINERS = ("bigram", "gpt2", "trl", "majority", "features", "weighted-gpt2") + + +def _parse_args() -> argparse.Namespace: + p = argparse.ArgumentParser(description=__doc__) + p.add_argument("trainer", choices=TRAINERS, help="Which training entrypoint to run") + p.add_argument("--hands", type=Path, default=HANDS_CLEAN) + p.add_argument("--group", default=None, help="WANDB_RUN_GROUP") + p.add_argument("--name", default=None, help="W&B run name override") + p.add_argument("--tags", default="", help="Comma-separated WANDB_TAGS") + p.add_argument( + "--require-wandb", + action="store_true", + help="Exit with an error if W&B is not configured (for catalog/sweep runs)", + ) + + # GPT-2 / TRL / weighted + p.add_argument("--n-positions", type=int, default=N_POSITIONS) + p.add_argument("--n-embd", type=int, default=N_EMBD) + p.add_argument("--n-layer", type=int, default=N_LAYER) + p.add_argument("--n-head", type=int, default=N_HEAD) + p.add_argument("--lr", type=float, default=GPT2_LR) + p.add_argument("--batch-size", type=int, default=GPT2_BATCH_SIZE) + p.add_argument("--epochs", type=int, default=GPT2_EPOCHS) + p.add_argument( + "--eval-max-examples", + type=int, + default=256, + help="Val hands for post-train action-type eval / charts (gpt2, weighted-gpt2)", + ) + p.add_argument( + "--skip-action-eval", + action="store_true", + help="Skip post-train generative action evaluation", + ) + + # Bigram + p.add_argument("--bigram-steps", type=int, default=BIGRAM_STEPS) + p.add_argument("--bigram-lr", type=float, default=BIGRAM_LR) + p.add_argument("--bigram-batch-size", type=int, default=BIGRAM_BATCH_SIZE) + + # Features approach + p.add_argument( + "--method", + choices=["rf", "logreg"], + default="rf", + help="features trainer: RandomForest or LogisticRegression", + ) + p.add_argument("--n-estimators", type=int, default=200) + p.add_argument("--max-depth", type=int, default=12) + p.add_argument("--C", type=float, default=1.0, help="LogReg inverse regularization") + return p.parse_args() + + +def _announce_wandb(require: bool) -> None: + ensure_wandb_project() + if wandb_is_configured(): + mode = os.environ.get("WANDB_MODE", "online (login/key)") + name = os.environ.get("WANDB_NAME", "(default trainer name)") + group = os.environ.get("WANDB_RUN_GROUP", "(none)") + tags = os.environ.get("WANDB_TAGS", "(none)") + entity = os.environ.get("WANDB_ENTITY", WANDB_ENTITY) + project = os.environ.get("WANDB_PROJECT", WANDB_PROJECT) + print( + f"W&B logging enabled — {entity}/{project} " + f"mode={mode} name={name} group={group} tags={tags}" + ) + return + + msg = ( + "W&B logging disabled — training will run without experiment tracking.\n" + " Enable: wandb login OR export WANDB_MODE=offline\n" + " Silence: export WANDB_MODE=disabled\n" + " Require: pass --require-wandb to fail if logging is unavailable\n" + f" Target: https://wandb.ai/{WANDB_ENTITY}/{WANDB_PROJECT}" + ) + print(msg, file=sys.stderr) + if require: + raise SystemExit( + "error: --require-wandb was set but W&B is not configured " + "(no API key / login, and WANDB_MODE is not offline/online/shared)" + ) + + +def _gpt2_hp(args: argparse.Namespace) -> GPT2Hyperparams: + if args.n_embd % args.n_head != 0: + raise SystemExit( + f"n_embd ({args.n_embd}) must be divisible by n_head ({args.n_head})" + ) + return GPT2Hyperparams( + n_positions=args.n_positions, + n_embd=args.n_embd, + n_layer=args.n_layer, + n_head=args.n_head, + learning_rate=args.lr, + batch_size=args.batch_size, + num_epochs=args.epochs, + ) + + +def main() -> None: + args = _parse_args() + if args.group: + os.environ["WANDB_RUN_GROUP"] = args.group + if args.tags: + os.environ["WANDB_TAGS"] = args.tags + if args.name: + os.environ["WANDB_NAME"] = args.name + + _announce_wandb(require=args.require_wandb) + + if args.trainer == "bigram": + from pokerai.training.train_bigram import main as train + + train( + hands_path=args.hands, + batch_size=args.bigram_batch_size, + n_steps=args.bigram_steps, + lr=args.bigram_lr, + ) + return + + if args.trainer == "majority": + from pokerai.training.train_majority import main as train + + train(hands_path=args.hands) + return + + if args.trainer == "features": + from pokerai.training.train_features import main as train + + train( + hands_path=args.hands, + method=args.method, + n_estimators=args.n_estimators, + max_depth=args.max_depth, + C=args.C, + ) + return + + hp = _gpt2_hp(args) + + if args.trainer == "gpt2": + from pokerai.training.train_gpt2 import main as train + + train( + hands_path=args.hands, + hp=hp, + class_weight=False, + run_action_eval=not args.skip_action_eval, + eval_max_examples=args.eval_max_examples, + ) + return + + if args.trainer == "weighted-gpt2": + from pokerai.training.train_gpt2 import main as train + from pokerai.training.train_weighted_gpt2 import MODEL_WEIGHTED_DIR + + train( + hands_path=args.hands, + output_dir=MODEL_WEIGHTED_DIR, + hp=hp, + class_weight=True, + run_action_eval=not args.skip_action_eval, + eval_max_examples=args.eval_max_examples, + ) + return + + # trl + from pokerai.training.train_trl import main as train + + train(hands_path=args.hands, hp=hp) + + +if __name__ == "__main__": + main() diff --git a/scripts/train_bigram.py b/scripts/train_bigram.py new file mode 100644 index 0000000..b41349f --- /dev/null +++ b/scripts/train_bigram.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Train the bigram baseline.""" + +from pokerai.training.train_bigram import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_features.py b/scripts/train_features.py new file mode 100644 index 0000000..8e814b4 --- /dev/null +++ b/scripts/train_features.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Structured-feature classifier (RF / LogReg).""" + +from pokerai.training.train_features import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_gpt2.py b/scripts/train_gpt2.py new file mode 100644 index 0000000..924dc7d --- /dev/null +++ b/scripts/train_gpt2.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Train GPT-2 with a raw PyTorch loop.""" + +from pokerai.training.train_gpt2 import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_majority.py b/scripts/train_majority.py new file mode 100644 index 0000000..54ff491 --- /dev/null +++ b/scripts/train_majority.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Majority-class action baseline.""" + +from pokerai.training.train_majority import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_tokenizer.py b/scripts/train_tokenizer.py new file mode 100644 index 0000000..91f5204 --- /dev/null +++ b/scripts/train_tokenizer.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Train the domain BPE tokenizer.""" + +from pokerai.data.tokenizer import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_trl.py b/scripts/train_trl.py new file mode 100644 index 0000000..2498db6 --- /dev/null +++ b/scripts/train_trl.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Train GPT-2 with Hugging Face TRL.""" + +from pokerai.training.train_trl import main + +if __name__ == "__main__": + main() diff --git a/scripts/train_weighted_gpt2.py b/scripts/train_weighted_gpt2.py new file mode 100644 index 0000000..02c71ab --- /dev/null +++ b/scripts/train_weighted_gpt2.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +"""Class-weighted GPT-2 training.""" + +from pokerai.training.train_weighted_gpt2 import main + +if __name__ == "__main__": + main() diff --git a/src/pokerai/__init__.py b/src/pokerai/__init__.py new file mode 100644 index 0000000..919e096 --- /dev/null +++ b/src/pokerai/__init__.py @@ -0,0 +1,3 @@ +"""Poker AI: train a small GPT-2 to predict poker actions from game state.""" + +__version__ = "0.2.0" diff --git a/src/pokerai/config.py b/src/pokerai/config.py new file mode 100644 index 0000000..0502020 --- /dev/null +++ b/src/pokerai/config.py @@ -0,0 +1,60 @@ +"""Shared paths and hyperparameters for the poker AI pipeline.""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +from pathlib import Path + +# Repo root: src/pokerai/config.py -> parents[2] +REPO_ROOT = Path(__file__).resolve().parents[2] + +DATA_DIR = REPO_ROOT / "data" +HANDS_RAW = DATA_DIR / "hands.txt" +HANDS_CLEAN = DATA_DIR / "hands_clean.txt" + +ARTIFACTS_DIR = REPO_ROOT / "artifacts" +TOKENIZER_DIR = ARTIFACTS_DIR / "tokenizer" +MODEL_GPT2_DIR = ARTIFACTS_DIR / "models" / "gpt2" +MODEL_TRL_DIR = ARTIFACTS_DIR / "models" / "gpt2_trl" +MODEL_BIGRAM_DIR = ARTIFACTS_DIR / "models" / "bigram" + +HF_DATASET = "SoelMgd/Poker_Dataset" +# https://wandb.ai/mooslin-university-of-wisconsin-madison/poker-ai +WANDB_ENTITY = "mooslin-university-of-wisconsin-madison" +WANDB_PROJECT = "poker-ai" + +# Special tokens — BOS and EOS must be distinct for clean generation stopping. +BOS_TOKEN = "<|startoftext|>" +EOS_TOKEN = "<|endoftext|>" +PAD_TOKEN = "<|pad|>" + +VOCAB_SIZE = 4000 +N_POSITIONS = 384 +N_EMBD = 256 +N_LAYER = 6 +N_HEAD = 8 + +TEST_SIZE = 0.05 +SPLIT_SEED = 42 + +GPT2_LR = 3e-4 +GPT2_BATCH_SIZE = 32 +GPT2_EPOCHS = 3 + +BIGRAM_LR = 1e-2 +BIGRAM_BATCH_SIZE = 4096 +BIGRAM_STEPS = 2000 + + +@dataclass(frozen=True) +class GPT2Hyperparams: + n_positions: int = N_POSITIONS + n_embd: int = N_EMBD + n_layer: int = N_LAYER + n_head: int = N_HEAD + learning_rate: float = GPT2_LR + batch_size: int = GPT2_BATCH_SIZE + num_epochs: int = GPT2_EPOCHS + + def as_dict(self) -> dict: + return asdict(self) diff --git a/src/pokerai/data/__init__.py b/src/pokerai/data/__init__.py new file mode 100644 index 0000000..4c5752c --- /dev/null +++ b/src/pokerai/data/__init__.py @@ -0,0 +1,164 @@ +"""Data loading, cleaning, and dataset helpers.""" + +from __future__ import annotations + +import re +from pathlib import Path + +from datasets import Dataset, DatasetDict, load_dataset + +from pokerai.config import HANDS_CLEAN, HANDS_RAW, HF_DATASET, SPLIT_SEED, TEST_SIZE + +# CALL 0BB is noise (no chips to put in); FOLD never carries an amount in this dataset. +_CALL_ZERO_BB = re.compile(r",CALL 0BB$") +_FOLD_WITH_AMOUNT = re.compile(r",FOLD\d+(\.\d+)?BB$") + +# Prompt / action separator used by every train and inference path. +ACTION_SEP = "," + + +def to_line(question: str, answer: str) -> str: + """Flatten multiline state into one line; join with comma for prompt/completion split.""" + context = " ".join(str(question).split()) + return f"{context}{ACTION_SEP}{str(answer).strip()}" + + +def clean_line(line: str) -> str: + """Normalize redundant action suffixes in serialized hands.""" + line = _FOLD_WITH_AMOUNT.sub(",FOLD", line) + line = _CALL_ZERO_BB.sub(",CALL", line) + return line + + +def fetch_hands(output_path: Path = HANDS_RAW) -> list[str]: + """Download SoelMgd/Poker_Dataset and write one hand per line.""" + ds = load_dataset(HF_DATASET) + frames = [] + for split in ds: + frames.append(ds[split]) + combined = frames[0] + for frame in frames[1:]: + combined = concatenate_datasets_safe(combined, frame) + + lines = [to_line(q, a) for q, a in zip(combined["question"], combined["answer"])] + output_path.parent.mkdir(parents=True, exist_ok=True) + output_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + return lines + + +def concatenate_datasets_safe(a: Dataset, b: Dataset) -> Dataset: + from datasets import concatenate_datasets + + return concatenate_datasets([a, b]) + + +def write_cleaned(lines: list[str], output_path: Path = HANDS_CLEAN) -> list[str]: + cleaned = [clean_line(line) for line in lines] + output_path.parent.mkdir(parents=True, exist_ok=True) + output_path.write_text("\n".join(cleaned) + "\n", encoding="utf-8") + return cleaned + + +def load_text_split( + path: Path = HANDS_CLEAN, + test_size: float = TEST_SIZE, + seed: int = SPLIT_SEED, +) -> DatasetDict: + """Load cleaned hands and split into train/test. + + Skips blank lines. Raises a clear error if the file is missing or empty + after filtering. + """ + if not path.exists(): + raise FileNotFoundError( + f"Missing {path}. Run: python scripts/prepare_data.py" + ) + if path.stat().st_size == 0: + raise ValueError( + f"{path} is empty. Re-run: python scripts/prepare_data.py" + ) + + raw = load_dataset("text", data_files={"train": str(path)})["train"] + raw = raw.filter(lambda ex: bool(ex["text"] and ex["text"].strip())) + if len(raw) == 0: + raise ValueError( + f"{path} has no non-empty lines. Re-run: python scripts/prepare_data.py" + ) + return raw.train_test_split(test_size=test_size, seed=seed) + + +def split_prompt_completion(text: str) -> tuple[str, str]: + """Split at the last comma into (prompt including comma, completion).""" + cut = text.rfind(ACTION_SEP) + 1 + if cut <= 0: + raise ValueError(f"Hand line has no comma separator: {text[:80]!r}") + return text[:cut], text[cut:] + + +def encode_with_mask( + text: str, + tokenizer, + max_length: int, +) -> tuple[list[int], list[int]]: + """Tokenize the full hand once and mask labels up to the action boundary. + + Locates the prompt/completion cut from character offsets on a single + tokenization pass. Tokenizing the prompt alone can disagree with joint + BPE merges, which silently shifts the supervised region. + + Context tokens (and BOS) get label -100; action tokens and EOS are supervised. + Long sequences keep the end (action) by dropping tokens from the front. + """ + prompt_text, _completion_text = split_prompt_completion(text) + cut = len(prompt_text) + + bos = tokenizer.bos_token_id + eos = tokenizer.eos_token_id + if bos is None or eos is None: + raise ValueError("Tokenizer must define bos_token_id and eos_token_id") + + # Prefer offset mapping (fast tokenizers). Fall back to separate encodes. + body_ids: list[int] + prompt_body_len: int + try: + enc = tokenizer( + text, + add_special_tokens=False, + return_offsets_mapping=True, + ) + body_ids = list(enc["input_ids"]) + offsets = enc.get("offset_mapping") + if not offsets: + raise TypeError("tokenizer did not return offset_mapping") + prompt_body_len = next( + (i for i, (start, _end) in enumerate(offsets) if start >= cut), + len(body_ids), + ) + except (TypeError, ValueError, AttributeError): + prompt_ids = tokenizer(prompt_text, add_special_tokens=False)["input_ids"] + completion_ids = tokenizer( + text[cut:], add_special_tokens=False + )["input_ids"] + body_ids = list(prompt_ids) + list(completion_ids) + prompt_body_len = len(prompt_ids) + + full_ids = [bos] + body_ids + [eos] + # Supervised region starts after BOS + prompt body tokens. + prompt_len = 1 + prompt_body_len + + if len(full_ids) > max_length: + drop = len(full_ids) - max_length + full_ids = full_ids[drop:] + prompt_len = max(0, prompt_len - drop) + + labels = [-100] * len(full_ids) + for i in range(prompt_len, len(full_ids)): + labels[i] = full_ids[i] + + if all(t == -100 for t in labels): + raise ValueError( + "No supervised action tokens after encoding/truncation; " + f"hand may be malformed or longer than max_length={max_length}: " + f"{text[:80]!r}" + ) + return full_ids, labels diff --git a/src/pokerai/data/prepare.py b/src/pokerai/data/prepare.py new file mode 100644 index 0000000..e6e7ba6 --- /dev/null +++ b/src/pokerai/data/prepare.py @@ -0,0 +1,20 @@ +"""Prepare hand histories from Hugging Face and write cleaned text files.""" + +from __future__ import annotations + +from pokerai.config import HANDS_CLEAN, HANDS_RAW +from pokerai.data import fetch_hands, write_cleaned + + +def main() -> None: + lines = fetch_hands(HANDS_RAW) + print(f"Wrote {len(lines)} hands to {HANDS_RAW}") + + cleaned = write_cleaned(lines, HANDS_CLEAN) + changed = sum(1 for a, b in zip(lines, cleaned) if a != b) + print(f"Modified {changed} / {len(lines)} lines -> {HANDS_CLEAN}") + print("Example:", cleaned[0][:160], "...") + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/data/tokenizer.py b/src/pokerai/data/tokenizer.py new file mode 100644 index 0000000..12b3668 --- /dev/null +++ b/src/pokerai/data/tokenizer.py @@ -0,0 +1,89 @@ +"""Train a domain BPE tokenizer on cleaned poker hand histories.""" + +from __future__ import annotations + +from pathlib import Path + +from tokenizers import ByteLevelBPETokenizer +from transformers import PreTrainedTokenizerFast + +from pokerai.config import ( + BOS_TOKEN, + EOS_TOKEN, + HANDS_CLEAN, + HANDS_RAW, + N_POSITIONS, + PAD_TOKEN, + TOKENIZER_DIR, + VOCAB_SIZE, +) + + +def train_tokenizer( + hands_path: Path | None = None, + output_dir: Path = TOKENIZER_DIR, + vocab_size: int = VOCAB_SIZE, +) -> PreTrainedTokenizerFast: + # Prefer cleaned hands so merges match what models actually train on. + if hands_path is None: + if HANDS_CLEAN.exists(): + hands_path = HANDS_CLEAN + elif HANDS_RAW.exists(): + hands_path = HANDS_RAW + else: + raise FileNotFoundError( + f"Missing {HANDS_CLEAN} (and {HANDS_RAW}). " + "Run: python scripts/prepare_data.py" + ) + + if not hands_path.exists(): + raise FileNotFoundError( + f"Missing {hands_path}. Run: python scripts/prepare_data.py" + ) + + bpe = ByteLevelBPETokenizer() + bpe.train( + files=[str(hands_path)], + vocab_size=vocab_size, + min_frequency=2, + special_tokens=[BOS_TOKEN, EOS_TOKEN, PAD_TOKEN], + ) + + output_dir.mkdir(parents=True, exist_ok=True) + tokenizer_json = output_dir / "tokenizer.json" + bpe.save(str(tokenizer_json)) + + hf_tokenizer = PreTrainedTokenizerFast(tokenizer_file=str(tokenizer_json)) + hf_tokenizer.add_special_tokens( + { + "pad_token": PAD_TOKEN, + "bos_token": BOS_TOKEN, + "eos_token": EOS_TOKEN, + } + ) + hf_tokenizer.model_max_length = N_POSITIONS + hf_tokenizer.save_pretrained(str(output_dir)) + return hf_tokenizer + + +def main() -> None: + hands_path = HANDS_CLEAN if HANDS_CLEAN.exists() else HANDS_RAW + tokenizer = train_tokenizer(hands_path=hands_path) + lines = hands_path.read_text(encoding="utf-8").splitlines() + if not lines: + raise ValueError(f"{hands_path} has no lines to inspect") + sample = lines[0] + ids = tokenizer(sample)["input_ids"] + print(f"Tokenizer corpus: {hands_path}") + print("Sample hand:", sample[:120], "...") + print("Token count (sample):", len(ids)) + + lengths = [len(tokenizer(line)["input_ids"]) for line in lines] + print("Max tokens in a hand:", max(lengths)) + print("Avg tokens per hand:", sum(lengths) / len(lengths)) + print(f"Saved tokenizer to {TOKENIZER_DIR}") + print(f"Vocab size: {len(tokenizer)}") + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/eval/__init__.py b/src/pokerai/eval/__init__.py new file mode 100644 index 0000000..ef6a91e --- /dev/null +++ b/src/pokerai/eval/__init__.py @@ -0,0 +1,19 @@ +"""Evaluation: action metrics, charts, and feature importance.""" + +from pokerai.eval.actions import ACTION_TYPES, action_type, action_type_counts +from pokerai.eval.importance import occlude_region, occlusion_importance +from pokerai.eval.metrics import ActionMetrics, compute_action_metrics +from pokerai.eval.report import run_lm_evaluation_report +from pokerai.eval.visualize import log_action_charts_to_wandb + +__all__ = [ + "ACTION_TYPES", + "ActionMetrics", + "action_type", + "action_type_counts", + "compute_action_metrics", + "log_action_charts_to_wandb", + "occlude_region", + "occlusion_importance", + "run_lm_evaluation_report", +] diff --git a/src/pokerai/eval/actions.py b/src/pokerai/eval/actions.py new file mode 100644 index 0000000..9553721 --- /dev/null +++ b/src/pokerai/eval/actions.py @@ -0,0 +1,34 @@ +"""Action-type utilities for poker decision evaluation.""" + +from __future__ import annotations + +import re +from collections import Counter + +# Canonical decision classes we chart and score. +ACTION_TYPES: tuple[str, ...] = ("FOLD", "CALL", "RAISE", "CHECK", "BET", "OTHER") + +_ACTION_RE = re.compile( + r"^\s*(FOLD|CHECK|CALL|RAISE|BET|ALL[\s-]?IN)\b", + re.IGNORECASE, +) + + +def action_type(text: str) -> str: + """Map a free-form action string to a coarse class.""" + if not text or not str(text).strip(): + return "OTHER" + m = _ACTION_RE.match(str(text).strip()) + if not m: + return "OTHER" + raw = m.group(1).upper().replace(" ", "").replace("-", "") + if raw == "ALLIN": + return "RAISE" + if raw in ACTION_TYPES: + return raw + return "OTHER" + + +def action_type_counts(actions: list[str]) -> dict[str, int]: + counts = Counter(action_type(a) for a in actions) + return {k: int(counts.get(k, 0)) for k in ACTION_TYPES} diff --git a/src/pokerai/eval/importance.py b/src/pokerai/eval/importance.py new file mode 100644 index 0000000..e333a64 --- /dev/null +++ b/src/pokerai/eval/importance.py @@ -0,0 +1,44 @@ +"""Occlusion-based feature importance over serialized poker states.""" + +from __future__ import annotations + +import re +from collections.abc import Callable + +# Named spans we ablate from the prompt (game state before the decision comma). +_REGION_PATTERNS: dict[str, re.Pattern[str]] = { + "table_config": re.compile(r"\[TABLE_CONFIGURATION\][^\[\]]*"), + "stacks": re.compile(r"\[STACKS\][^\[\]]*"), + "pot": re.compile(r"\bPOT=\d+(\.\d+)?BB\b"), + "preflop": re.compile(r"\[PREFLOP\][^\[\]]*"), + "flop": re.compile(r"\[FLOP\][^\[\]]*"), + "turn": re.compile(r"\[TURN\][^\[\]]*"), + "river": re.compile(r"\[RIVER\][^\[\]]*"), + "hole_cards": re.compile(r"\[[A-Za-z0-9]{2}(?:\s+[A-Za-z0-9]{2})*\]"), +} + + +def occlude_region(prompt: str, region: str) -> str: + """Replace a named region with a placeholder so length stays nonzero.""" + pattern = _REGION_PATTERNS.get(region) + if pattern is None: + raise KeyError(f"Unknown region {region!r}; choose from {list(_REGION_PATTERNS)}") + return pattern.sub(f"[{region.upper()}_MASKED]", prompt) + + +def occlusion_importance( + prompts: list[str], + score_fn: Callable[[list[str]], float], + regions: list[str] | None = None, +) -> dict[str, float]: + """Importance = baseline_score - score_after_occlusion (higher ⇒ more important). + + ``score_fn`` should return a scalar where higher is better (e.g. accuracy). + """ + regions = regions or list(_REGION_PATTERNS) + baseline = score_fn(prompts) + scores: dict[str, float] = {} + for region in regions: + ablated = [occlude_region(p, region) for p in prompts] + scores[region] = float(baseline - score_fn(ablated)) + return scores diff --git a/src/pokerai/eval/metrics.py b/src/pokerai/eval/metrics.py new file mode 100644 index 0000000..073f334 --- /dev/null +++ b/src/pokerai/eval/metrics.py @@ -0,0 +1,92 @@ +"""Classification metrics over coarse action types.""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass + +from pokerai.eval.actions import ACTION_TYPES, action_type + + +@dataclass(frozen=True) +class ClassScores: + precision: float + recall: float + f1: float + support: int + + +@dataclass(frozen=True) +class ActionMetrics: + accuracy: float + macro_f1: float + per_class: dict[str, ClassScores] + confusion: list[list[int]] # rows = true, cols = pred, order ACTION_TYPES + n: int + + def as_dict(self) -> dict: + return { + "accuracy": self.accuracy, + "macro_f1": self.macro_f1, + "n": self.n, + "per_class": {k: asdict(v) for k, v in self.per_class.items()}, + "confusion": self.confusion, + "labels": list(ACTION_TYPES), + } + + def flat_wandb_metrics(self, prefix: str = "action") -> dict[str, float]: + out: dict[str, float] = { + f"{prefix}/accuracy": self.accuracy, + f"{prefix}/macro_f1": self.macro_f1, + f"{prefix}/n": float(self.n), + } + for name, scores in self.per_class.items(): + key = name.lower() + out[f"{prefix}/{key}/precision"] = scores.precision + out[f"{prefix}/{key}/recall"] = scores.recall + out[f"{prefix}/{key}/f1"] = scores.f1 + out[f"{prefix}/{key}/support"] = float(scores.support) + return out + + +def compute_action_metrics( + y_true: list[str], + y_pred: list[str], +) -> ActionMetrics: + if len(y_true) != len(y_pred): + raise ValueError("y_true and y_pred must have the same length") + true_t = [action_type(y) for y in y_true] + pred_t = [action_type(y) for y in y_pred] + n = len(true_t) + index = {label: i for i, label in enumerate(ACTION_TYPES)} + conf = [[0 for _ in ACTION_TYPES] for _ in ACTION_TYPES] + correct = 0 + for t, p in zip(true_t, pred_t): + conf[index[t]][index[p]] += 1 + if t == p: + correct += 1 + + per_class: dict[str, ClassScores] = {} + f1s: list[float] = [] + for i, label in enumerate(ACTION_TYPES): + tp = conf[i][i] + fp = sum(conf[r][i] for r in range(len(ACTION_TYPES))) - tp + fn = sum(conf[i]) - tp + support = sum(conf[i]) + precision = tp / (tp + fp) if (tp + fp) else 0.0 + recall = tp / (tp + fn) if (tp + fn) else 0.0 + f1 = ( + 2 * precision * recall / (precision + recall) + if (precision + recall) + else 0.0 + ) + per_class[label] = ClassScores(precision, recall, f1, support) + if support > 0: + f1s.append(f1) + + return ActionMetrics( + accuracy=correct / n if n else 0.0, + macro_f1=sum(f1s) / len(f1s) if f1s else 0.0, + per_class=per_class, + confusion=conf, + n=n, + ) diff --git a/src/pokerai/eval/report.py b/src/pokerai/eval/report.py new file mode 100644 index 0000000..0307c93 --- /dev/null +++ b/src/pokerai/eval/report.py @@ -0,0 +1,143 @@ +"""Evaluate generative / classifier models on action-type metrics.""" + +from __future__ import annotations + +from pathlib import Path + +import torch +from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast + +from pokerai.config import ARTIFACTS_DIR, N_POSITIONS +from pokerai.data import split_prompt_completion +from pokerai.eval.importance import occlusion_importance +from pokerai.eval.metrics import ActionMetrics, compute_action_metrics +from pokerai.eval.visualize import log_action_charts_to_wandb, save_confusion_png, save_importance_png, save_per_class_f1_png +from pokerai.inference.predict import predict_action +from pokerai.training import get_device + + +def evaluate_generations( + model: GPT2LMHeadModel, + tokenizer: PreTrainedTokenizerFast, + lines: list[str], + *, + max_examples: int | None = None, + max_new_tokens: int = 16, +) -> tuple[ActionMetrics, list[str], list[str], list[str]]: + """Greedy-generate actions for each hand line; return metrics + parallel lists.""" + subset = lines[:max_examples] if max_examples is not None else lines + prompts: list[str] = [] + y_true: list[str] = [] + y_pred: list[str] = [] + for line in subset: + prompt, completion = split_prompt_completion(line) + prompts.append(prompt) + y_true.append(completion) + y_pred.append( + predict_action(prompt, model, tokenizer, max_new_tokens=max_new_tokens) + ) + metrics = compute_action_metrics(y_true, y_pred) + return metrics, prompts, y_true, y_pred + + +def teacher_forced_action_type_accuracy( + model: GPT2LMHeadModel, + tokenizer: PreTrainedTokenizerFast, + lines: list[str], + *, + max_examples: int | None = 256, +) -> float: + """Accuracy of the first supervised action token under teacher forcing.""" + from pokerai.data import encode_with_mask + from pokerai.eval.actions import action_type + + device = get_device() + model.eval() + model.to(device) + subset = lines[:max_examples] if max_examples is not None else lines + correct = total = 0 + with torch.no_grad(): + for line in subset: + ids, labels = encode_with_mask(line, tokenizer, N_POSITIONS) + # First supervised index + try: + start = next(i for i, t in enumerate(labels) if t != -100) + except StopIteration: + continue + if start == 0: + continue + tensor = torch.tensor([ids[:start]], device=device) + logits = model(tensor).logits[0, -1] + pred_id = int(logits.argmax()) + true_id = ids[start] + pred_tok = tokenizer.decode([pred_id]).strip() + true_tok = tokenizer.decode([true_id]).strip() + # Compare coarse type when tokens are action words; else exact token. + if action_type(pred_tok) == action_type(true_tok) and action_type(true_tok) != "OTHER": + correct += 1 + elif pred_id == true_id: + correct += 1 + total += 1 + return correct / total if total else 0.0 + + +def run_lm_evaluation_report( + model: GPT2LMHeadModel, + tokenizer: PreTrainedTokenizerFast, + lines: list[str], + *, + report_dir: Path | None = None, + max_examples: int = 256, + max_importance: int = 64, + log_wandb: bool = True, +) -> dict: + """Full eval: action metrics, charts, and occlusion importance.""" + report_dir = Path(report_dir or (ARTIFACTS_DIR / "reports" / "lm_eval")) + report_dir.mkdir(parents=True, exist_ok=True) + + metrics, prompts, y_true, y_pred = evaluate_generations( + model, tokenizer, lines, max_examples=max_examples + ) + + def _acc_on_prompts(ps: list[str]) -> float: + # Score = fraction of generations matching true action type on the paired labels. + preds = [ + predict_action(p, model, tokenizer) for p in ps + ] + # Align with the same slice of y_true + m = compute_action_metrics(y_true[: len(preds)], preds) + return m.accuracy + + importance: dict[str, float] = {} + if prompts: + n_imp = min(max_importance, len(prompts)) + importance = occlusion_importance( + prompts[:n_imp], + lambda ps: _acc_on_prompts(ps), + ) + + save_confusion_png(metrics, report_dir / "confusion.png") + save_per_class_f1_png(metrics, report_dir / "per_class_f1.png") + if importance: + save_importance_png(importance, report_dir / "feature_importance.png") + + if log_wandb: + log_action_charts_to_wandb( + metrics, + y_true=y_true, + y_pred=y_pred, + importance=importance, + artifact_dir=report_dir, + prefix="eval", + ) + + summary = { + "metrics": metrics.as_dict(), + "importance": importance, + "report_dir": str(report_dir), + "max_examples": max_examples, + } + (report_dir / "summary.json").write_text( + __import__("json").dumps(summary, indent=2), encoding="utf-8" + ) + return summary diff --git a/src/pokerai/eval/visualize.py b/src/pokerai/eval/visualize.py new file mode 100644 index 0000000..e6328de --- /dev/null +++ b/src/pokerai/eval/visualize.py @@ -0,0 +1,161 @@ +"""Chart helpers for local artifacts and W&B.""" + +from __future__ import annotations + +from pathlib import Path + +from pokerai.eval.actions import ACTION_TYPES +from pokerai.eval.metrics import ActionMetrics + + +def save_confusion_png(metrics: ActionMetrics, path: Path, title: str = "Action confusion") -> Path: + """Save a confusion-matrix heatmap PNG (requires matplotlib).""" + import matplotlib + + matplotlib.use("Agg") + import matplotlib.pyplot as plt + import numpy as np + + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + mat = np.array(metrics.confusion, dtype=float) + # Row-normalize for readability when FOLD dominates. + row_sums = mat.sum(axis=1, keepdims=True) + norm = np.divide(mat, row_sums, out=np.zeros_like(mat), where=row_sums > 0) + + fig, ax = plt.subplots(figsize=(7, 6)) + im = ax.imshow(norm, cmap="Blues", vmin=0, vmax=1) + ax.set_xticks(range(len(ACTION_TYPES))) + ax.set_yticks(range(len(ACTION_TYPES))) + ax.set_xticklabels(ACTION_TYPES, rotation=45, ha="right") + ax.set_yticklabels(ACTION_TYPES) + ax.set_xlabel("Predicted") + ax.set_ylabel("True") + ax.set_title(title) + for i in range(len(ACTION_TYPES)): + for j in range(len(ACTION_TYPES)): + ax.text(j, i, int(mat[i, j]), ha="center", va="center", color="black", fontsize=8) + fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04) + fig.tight_layout() + fig.savefig(path, dpi=140) + plt.close(fig) + return path + + +def save_importance_png( + scores: dict[str, float], + path: Path, + title: str = "Feature importance", +) -> Path: + import matplotlib + + matplotlib.use("Agg") + import matplotlib.pyplot as plt + + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + labels = list(scores.keys()) + values = [scores[k] for k in labels] + order = sorted(range(len(values)), key=lambda i: values[i], reverse=True) + labels = [labels[i] for i in order] + values = [values[i] for i in order] + + fig, ax = plt.subplots(figsize=(8, max(3, 0.4 * len(labels) + 1))) + ax.barh(labels[::-1], values[::-1], color="#2a6f97") + ax.set_xlabel("Importance / Δ metric") + ax.set_title(title) + fig.tight_layout() + fig.savefig(path, dpi=140) + plt.close(fig) + return path + + +def save_per_class_f1_png(metrics: ActionMetrics, path: Path, title: str = "Per-action F1") -> Path: + import matplotlib + + matplotlib.use("Agg") + import matplotlib.pyplot as plt + + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + labels = list(ACTION_TYPES) + values = [metrics.per_class[l].f1 for l in labels] + supports = [metrics.per_class[l].support for l in labels] + + fig, ax = plt.subplots(figsize=(8, 4)) + bars = ax.bar(labels, values, color="#e76f51") + ax.set_ylim(0, 1.05) + ax.set_ylabel("F1") + ax.set_title(title) + for bar, support in zip(bars, supports): + ax.text( + bar.get_x() + bar.get_width() / 2, + bar.get_height() + 0.02, + f"n={support}", + ha="center", + va="bottom", + fontsize=8, + ) + fig.tight_layout() + fig.savefig(path, dpi=140) + plt.close(fig) + return path + + +def log_action_charts_to_wandb( + metrics: ActionMetrics, + *, + y_true: list[str] | None = None, + y_pred: list[str] | None = None, + importance: dict[str, float] | None = None, + artifact_dir: Path | None = None, + prefix: str = "eval", +) -> dict: + """Log scalars + confusion/importance plots to the active W&B run (if any).""" + payload = metrics.flat_wandb_metrics(prefix=f"{prefix}/action") + try: + import wandb + except ImportError: + return payload + if wandb.run is None: + return payload + + from pokerai.eval.actions import action_type + + if y_true is not None and y_pred is not None: + payload[f"{prefix}/confusion"] = wandb.plot.confusion_matrix( + preds=[action_type(p) for p in y_pred], + y_true=[action_type(t) for t in y_true], + class_names=list(ACTION_TYPES), + title="Action-type confusion", + ) + + table = wandb.Table(columns=["action", "precision", "recall", "f1", "support"]) + for name, scores in metrics.per_class.items(): + table.add_data(name, scores.precision, scores.recall, scores.f1, scores.support) + payload[f"{prefix}/per_class_table"] = table + + if artifact_dir is not None: + artifact_dir = Path(artifact_dir) + cm_path = save_confusion_png(metrics, artifact_dir / "confusion.png") + f1_path = save_per_class_f1_png(metrics, artifact_dir / "per_class_f1.png") + payload[f"{prefix}/confusion_image"] = wandb.Image(str(cm_path)) + payload[f"{prefix}/per_class_f1_image"] = wandb.Image(str(f1_path)) + if importance: + imp_path = save_importance_png( + importance, artifact_dir / "feature_importance.png" + ) + payload[f"{prefix}/feature_importance_image"] = wandb.Image(str(imp_path)) + + if importance: + imp_table = wandb.Table(columns=["feature", "importance"]) + for k, v in sorted(importance.items(), key=lambda kv: -kv[1]): + imp_table.add_data(k, v) + payload[f"{prefix}/feature_importance_table"] = imp_table + for k, v in importance.items(): + payload[f"{prefix}/importance/{k}"] = float(v) + + wandb.log(payload) + for k, v in metrics.flat_wandb_metrics(prefix=f"{prefix}/action").items(): + wandb.summary[k] = v + return payload diff --git a/src/pokerai/features/__init__.py b/src/pokerai/features/__init__.py new file mode 100644 index 0000000..1ed1bd5 --- /dev/null +++ b/src/pokerai/features/__init__.py @@ -0,0 +1,125 @@ +"""Structured feature extraction from serialized poker hands.""" + +from __future__ import annotations + +import re +from dataclasses import asdict, dataclass + +from pokerai.data import split_prompt_completion +from pokerai.eval.actions import action_type + +_POT = re.compile(r"POT=(\d+(?:\.\d+)?)BB") +_STACK = re.compile(r"P\d+:\s*(\d+(?:\.\d+)?)BB") +_HOLE = re.compile(r"\[([A-Za-z0-9]{2})\s+([A-Za-z0-9]{2})\]") +_STREETS = ("PREFLOP", "FLOP", "TURN", "RIVER") +_RANK = { + "2": 2, + "3": 3, + "4": 4, + "5": 5, + "6": 6, + "7": 7, + "8": 8, + "9": 9, + "T": 10, + "J": 11, + "Q": 12, + "K": 13, + "A": 14, +} + + +@dataclass(frozen=True) +class HandFeatures: + pot_bb: float + n_stacks: int + max_stack_bb: float + min_stack_bb: float + mean_stack_bb: float + street_preflop: int + street_flop: int + street_turn: int + street_river: int + n_raises: int + n_calls: int + n_bets: int + n_checks: int + n_folds_hist: int + facing_aggression: int + has_hole_cards: int + hole_high: int + hole_low: int + hole_suited: int + hole_pair: int + prompt_chars: int + + def vector(self) -> list[float]: + return [float(v) for v in asdict(self).values()] + + +def extract_features(prompt: str) -> HandFeatures: + """Extract numeric features from a game-state prompt (no trailing action).""" + text = prompt if not prompt.endswith(",") else prompt + pot_m = _POT.search(text) + pot = float(pot_m.group(1)) if pot_m else 0.0 + stacks = [float(x) for x in _STACK.findall(text)] + street_flags = {f"street_{s.lower()}": int(f"[{s}]" in text) for s in _STREETS} + # Prefer the last street present as "current". + for s in reversed(_STREETS): + if f"[{s}]" in text: + for k in street_flags: + street_flags[k] = 0 + street_flags[f"street_{s.lower()}"] = 1 + break + + hist = text.upper() + n_raises = len(re.findall(r"\bRAISE\b", hist)) + n_calls = len(re.findall(r"\bCALL\b", hist)) + n_bets = len(re.findall(r"\bBET\b", hist)) + n_checks = len(re.findall(r"\bCHECK\b", hist)) + n_folds = len(re.findall(r"\bFOLD\b", hist)) + facing = int(n_raises + n_bets > 0) + + hole = _HOLE.search(text) + has_hole = int(hole is not None) + hole_high = hole_low = hole_suited = hole_pair = 0 + if hole: + c1, c2 = hole.group(1), hole.group(2) + r1, r2 = _RANK.get(c1[0].upper(), 0), _RANK.get(c2[0].upper(), 0) + hole_high, hole_low = max(r1, r2), min(r1, r2) + hole_suited = int(len(c1) > 1 and len(c2) > 1 and c1[1] == c2[1]) + hole_pair = int(r1 == r2 and r1 > 0) + + return HandFeatures( + pot_bb=pot, + n_stacks=len(stacks), + max_stack_bb=max(stacks) if stacks else 0.0, + min_stack_bb=min(stacks) if stacks else 0.0, + mean_stack_bb=(sum(stacks) / len(stacks)) if stacks else 0.0, + street_preflop=street_flags["street_preflop"], + street_flop=street_flags["street_flop"], + street_turn=street_flags["street_turn"], + street_river=street_flags["street_river"], + n_raises=n_raises, + n_calls=n_calls, + n_bets=n_bets, + n_checks=n_checks, + n_folds_hist=n_folds, + facing_aggression=facing, + has_hole_cards=has_hole, + hole_high=hole_high, + hole_low=hole_low, + hole_suited=hole_suited, + hole_pair=hole_pair, + prompt_chars=len(text), + ) + + +def features_and_label(line: str) -> tuple[list[float], str]: + prompt, completion = split_prompt_completion(line) + return extract_features(prompt).vector(), action_type(completion) + + +# Fix HandFeatures.names() to not need a weird constructor +def feature_names() -> list[str]: + return list(HandFeatures.__dataclass_fields__.keys()) diff --git a/src/pokerai/inference/__init__.py b/src/pokerai/inference/__init__.py new file mode 100644 index 0000000..773e0c7 --- /dev/null +++ b/src/pokerai/inference/__init__.py @@ -0,0 +1,5 @@ +"""Inference helpers for poker action prediction.""" + +from pokerai.inference.predict import load_model, main, predict_action + +__all__ = ["load_model", "main", "predict_action"] diff --git a/src/pokerai/inference/predict.py b/src/pokerai/inference/predict.py new file mode 100644 index 0000000..a3a2d94 --- /dev/null +++ b/src/pokerai/inference/predict.py @@ -0,0 +1,87 @@ +"""Generate an action given a serialized poker game state.""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +import torch +from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast + +from pokerai.config import MODEL_GPT2_DIR, MODEL_TRL_DIR, N_POSITIONS +from pokerai.training import get_device + + +def load_model( + model_dir: Path, +) -> tuple[GPT2LMHeadModel, PreTrainedTokenizerFast]: + if not model_dir.exists(): + raise FileNotFoundError( + f"No model at {model_dir}. Train first with scripts/train_gpt2.py " + "or scripts/train_trl.py" + ) + tokenizer = PreTrainedTokenizerFast.from_pretrained(str(model_dir)) + model = GPT2LMHeadModel.from_pretrained(str(model_dir)) + return model, tokenizer + + +def predict_action( + state: str, + model: GPT2LMHeadModel, + tokenizer: PreTrainedTokenizerFast, + max_new_tokens: int = 16, +) -> str: + """Predict the hero action for a game-state string (no trailing action).""" + prompt = state if state.endswith(",") else state + "," + device = get_device() + model.to(device) + model.eval() + + bos = tokenizer.bos_token_id + if bos is None: + raise ValueError("Tokenizer is missing bos_token_id") + input_ids = [bos] + tokenizer(prompt, add_special_tokens=False)["input_ids"] + budget = N_POSITIONS - max_new_tokens + if budget < 1: + raise ValueError( + f"max_new_tokens={max_new_tokens} leaves no room under N_POSITIONS={N_POSITIONS}" + ) + if len(input_ids) > budget: + input_ids = input_ids[-budget:] + + tensor = torch.tensor([input_ids], device=device) + with torch.no_grad(): + out = model.generate( + tensor, + max_new_tokens=max_new_tokens, + eos_token_id=tokenizer.eos_token_id, + pad_token_id=tokenizer.pad_token_id, + do_sample=False, + ) + generated = out[0, len(input_ids) :].tolist() + text = tokenizer.decode(generated, skip_special_tokens=True).strip() + return text + + +def main() -> None: + parser = argparse.ArgumentParser(description="Predict a poker action from game state") + parser.add_argument( + "state", + nargs="?", + help="Serialized game state (prompt). If omitted, reads stdin.", + ) + parser.add_argument( + "--model", + type=Path, + default=MODEL_TRL_DIR if MODEL_TRL_DIR.exists() else MODEL_GPT2_DIR, + help="Directory with saved model + tokenizer", + ) + args = parser.parse_args() + state = args.state or input("Game state: ").strip() + model, tokenizer = load_model(args.model) + action = predict_action(state, model, tokenizer) + print(action) + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/models/__init__.py b/src/pokerai/models/__init__.py new file mode 100644 index 0000000..7fc86e3 --- /dev/null +++ b/src/pokerai/models/__init__.py @@ -0,0 +1,46 @@ +"""Model constructors.""" + +from __future__ import annotations + +import torch.nn as nn +from transformers import GPT2Config, GPT2LMHeadModel, PreTrainedTokenizerFast + +from pokerai.config import GPT2Hyperparams, N_EMBD, N_HEAD, N_LAYER, N_POSITIONS + + +class Bigram(nn.Module): + """Next-token table: row i = logits for the token after i.""" + + def __init__(self, vocab_size: int): + super().__init__() + self.table = nn.Embedding(vocab_size, vocab_size) + + def forward(self, idx): + return self.table(idx) + + +def build_gpt2( + tokenizer: PreTrainedTokenizerFast, + hp: GPT2Hyperparams | None = None, +) -> GPT2LMHeadModel: + hp = hp or GPT2Hyperparams() + config = GPT2Config( + vocab_size=len(tokenizer), + n_positions=hp.n_positions, + n_embd=hp.n_embd, + n_layer=hp.n_layer, + n_head=hp.n_head, + bos_token_id=tokenizer.bos_token_id, + eos_token_id=tokenizer.eos_token_id, + pad_token_id=tokenizer.pad_token_id, + ) + return GPT2LMHeadModel(config) + + +def default_gpt2_config_kwargs() -> dict: + return { + "n_positions": N_POSITIONS, + "n_embd": N_EMBD, + "n_layer": N_LAYER, + "n_head": N_HEAD, + } diff --git a/src/pokerai/training/__init__.py b/src/pokerai/training/__init__.py new file mode 100644 index 0000000..16c9f99 --- /dev/null +++ b/src/pokerai/training/__init__.py @@ -0,0 +1,56 @@ +"""Shared device / tokenizer / experiment-tracking helpers for training scripts.""" + +from __future__ import annotations + +import os +from pathlib import Path + +import torch +from transformers import PreTrainedTokenizerFast + +from pokerai.config import TOKENIZER_DIR, WANDB_ENTITY, WANDB_PROJECT + + +def get_device() -> str: + if torch.cuda.is_available(): + return "cuda" + if getattr(torch.backends, "mps", None) and torch.backends.mps.is_available(): + return "mps" + return "cpu" + + +def load_tokenizer(path: Path = TOKENIZER_DIR) -> PreTrainedTokenizerFast: + if not path.exists(): + raise FileNotFoundError( + f"Missing tokenizer at {path}. Run: python scripts/train_tokenizer.py" + ) + return PreTrainedTokenizerFast.from_pretrained(str(path)) + + +def wandb_is_configured() -> bool: + """True when W&B should run (explicit mode or logged-in API key).""" + mode = os.environ.get("WANDB_MODE", "").lower() + if mode == "disabled": + return False + if mode in ("offline", "online", "shared"): + return True + if os.environ.get("WANDB_API_KEY"): + return True + # Fall back to a saved login without forcing an interactive prompt. + try: + import wandb + + return bool(wandb.api.api_key) + except Exception: + return False + + +def wandb_report_to() -> str: + """Value for HF TrainingArguments.report_to — never force wandb.""" + return "wandb" if wandb_is_configured() else "none" + + +def ensure_wandb_project() -> None: + """Point W&B at the shared team project (entity/project).""" + os.environ.setdefault("WANDB_ENTITY", WANDB_ENTITY) + os.environ.setdefault("WANDB_PROJECT", WANDB_PROJECT) diff --git a/src/pokerai/training/train_bigram.py b/src/pokerai/training/train_bigram.py new file mode 100644 index 0000000..6ce7b3e --- /dev/null +++ b/src/pokerai/training/train_bigram.py @@ -0,0 +1,144 @@ +"""Bigram baseline trained only on action-token transitions (fairer vs GPT-2).""" + +from __future__ import annotations + +import math +import os +from pathlib import Path + +import torch +import torch.nn.functional as F + +from pokerai.config import ( + BIGRAM_BATCH_SIZE, + BIGRAM_LR, + BIGRAM_STEPS, + HANDS_CLEAN, + MODEL_BIGRAM_DIR, + WANDB_ENTITY, + WANDB_PROJECT, +) +from pokerai.data import encode_with_mask, load_text_split +from pokerai.models import Bigram +from pokerai.training import ( + ensure_wandb_project, + get_device, + load_tokenizer, + wandb_is_configured, +) + + +def _action_pairs(sequences: list[tuple[list[int], list[int]]]): + """Build (x, y) pairs only where the next token is supervised (action region).""" + xs, ys = [], [] + for ids, labels in sequences: + for i in range(len(ids) - 1): + if labels[i + 1] != -100: + xs.append(ids[i]) + ys.append(ids[i + 1]) + return torch.tensor(xs), torch.tensor(ys) + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_BIGRAM_DIR, + batch_size: int = BIGRAM_BATCH_SIZE, + n_steps: int = BIGRAM_STEPS, + lr: float = BIGRAM_LR, +) -> None: + tokenizer = load_tokenizer() + vocab_size = len(tokenizer) + max_length = tokenizer.model_max_length or 384 + + split = load_text_split(hands_path) + train_enc = [encode_with_mask(ex["text"], tokenizer, max_length) for ex in split["train"]] + val_enc = [encode_with_mask(ex["text"], tokenizer, max_length) for ex in split["test"]] + + train_x, train_y = _action_pairs(train_enc) + val_x, val_y = _action_pairs(val_enc) + print(f"Train pairs: {len(train_x):,} Val pairs: {len(val_x):,}") + if len(train_x) == 0: + raise RuntimeError( + "No supervised action pairs in the training split. " + "Check that hands contain a comma-separated action and that " + "the tokenizer was trained on the same cleaned corpus." + ) + if len(val_x) == 0: + raise RuntimeError("No supervised action pairs in the validation split.") + + device = get_device() + print("Training on:", device) + + use_wandb = wandb_is_configured() + if use_wandb: + import wandb + + ensure_wandb_project() + wandb.init( + entity=WANDB_ENTITY, + project=WANDB_PROJECT, + name=os.environ.get("WANDB_NAME", "bigram-baseline"), + config={ + "model": "bigram", + "vocab_size": vocab_size, + "batch_size": batch_size, + "n_steps": n_steps, + "learning_rate": lr, + "device": device, + "train_pairs": len(train_x), + "val_pairs": len(val_x), + "masked_action_only": True, + }, + ) + + model = Bigram(vocab_size).to(device) + optimizer = torch.optim.AdamW(model.parameters(), lr=lr) + train_x, train_y = train_x.to(device), train_y.to(device) + val_x, val_y = val_x.to(device), val_y.to(device) + + for step in range(n_steps): + idx = torch.randint(0, len(train_x), (batch_size,)) + xb, yb = train_x[idx], train_y[idx] + loss = F.cross_entropy(model(xb), yb) + optimizer.zero_grad() + loss.backward() + optimizer.step() + + if step % 200 == 0 or step == n_steps - 1: + with torch.no_grad(): + val_loss = F.cross_entropy(model(val_x), val_y) + print( + f"step {step:5d} | train loss {loss.item():.4f} | val loss {val_loss.item():.4f}" + ) + if use_wandb: + import wandb + + wandb.log( + {"train/loss": loss.item(), "val/loss": val_loss.item()}, + step=step, + ) + + with torch.no_grad(): + final_val_loss = F.cross_entropy(model(val_x), val_y).item() + + random_baseline = math.log(vocab_size) + print(f"\nRandom-guess baseline loss: {random_baseline:.4f}") + print(f"Bigram model final val loss: {final_val_loss:.4f}") + + output_dir.mkdir(parents=True, exist_ok=True) + torch.save( + {"state_dict": model.state_dict(), "vocab_size": vocab_size}, + output_dir / "bigram.pt", + ) + print(f"Saved to {output_dir / 'bigram.pt'}") + + if use_wandb: + import wandb + + wandb.summary["random_baseline_loss"] = random_baseline + wandb.summary["final_val_loss"] = final_val_loss + wandb.finish() + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/training/train_features.py b/src/pokerai/training/train_features.py new file mode 100644 index 0000000..2464dd1 --- /dev/null +++ b/src/pokerai/training/train_features.py @@ -0,0 +1,175 @@ +"""Structured-feature classifiers (LogReg / RandomForest) — non-LM approach.""" + +from __future__ import annotations + +import json +from pathlib import Path + +from pokerai.config import ARTIFACTS_DIR, HANDS_CLEAN, WANDB_ENTITY, WANDB_PROJECT +from pokerai.data import load_text_split +from pokerai.eval.actions import ACTION_TYPES +from pokerai.eval.metrics import compute_action_metrics +from pokerai.eval.visualize import log_action_charts_to_wandb +from pokerai.features import feature_names, features_and_label +from pokerai.training import ensure_wandb_project, wandb_is_configured + +MODEL_FEATURES_DIR = ARTIFACTS_DIR / "models" / "features" + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_FEATURES_DIR, + method: str = "rf", + n_estimators: int = 200, + max_depth: int | None = 12, + C: float = 1.0, +) -> None: + try: + from sklearn.ensemble import RandomForestClassifier + from sklearn.linear_model import LogisticRegression + from sklearn.pipeline import Pipeline + from sklearn.preprocessing import StandardScaler + except ImportError as exc: + raise SystemExit( + "scikit-learn is required for the features trainer. " + "Install with: pip install scikit-learn" + ) from exc + + split = load_text_split(hands_path) + x_train, y_train = [], [] + for ex in split["train"]: + x, y = features_and_label(ex["text"]) + x_train.append(x) + y_train.append(y) + x_val, y_val = [], [] + for ex in split["test"]: + x, y = features_and_label(ex["text"]) + x_val.append(x) + y_val.append(y) + + names = feature_names() + if method == "logreg": + clf = Pipeline( + [ + ("scaler", StandardScaler()), + ( + "model", + LogisticRegression( + max_iter=2000, + C=C, + class_weight="balanced", + multi_class="auto", + ), + ), + ] + ) + model_name = "logreg-features" + else: + clf = RandomForestClassifier( + n_estimators=n_estimators, + max_depth=max_depth, + class_weight="balanced_subsample", + random_state=42, + n_jobs=-1, + ) + model_name = "rf-features" + method = "rf" + + clf.fit(x_train, y_train) + y_pred = list(clf.predict(x_val)) + metrics = compute_action_metrics(y_val, y_pred) + + importance: dict[str, float] = {} + if method == "rf": + importances = getattr(clf, "feature_importances_", None) + if importances is not None: + importance = {n: float(v) for n, v in zip(names, importances)} + else: + # Mean absolute coefficient across classes (scaled features). + model = clf.named_steps["model"] + coef = getattr(model, "coef_", None) + if coef is not None: + import numpy as np + + mean_abs = np.mean(np.abs(coef), axis=0) + importance = {n: float(v) for n, v in zip(names, mean_abs)} + + print(f"Model: {model_name}") + print( + f"Val accuracy={metrics.accuracy:.4f} macro_f1={metrics.macro_f1:.4f} n={metrics.n}" + ) + for name, scores in metrics.per_class.items(): + print( + f" {name:6s} P={scores.precision:.3f} R={scores.recall:.3f} " + f"F1={scores.f1:.3f} support={scores.support}" + ) + if importance: + top = sorted(importance.items(), key=lambda kv: -kv[1])[:8] + print("Top features:", ", ".join(f"{k}={v:.3f}" for k, v in top)) + + output_dir.mkdir(parents=True, exist_ok=True) + report_dir = output_dir / "report" + try: + import joblib + + joblib.dump( + {"model": clf, "feature_names": names, "method": method, "labels": list(ACTION_TYPES)}, + output_dir / "features_model.joblib", + ) + except ImportError: + pass + + payload = { + "model": model_name, + "method": method, + "metrics": metrics.as_dict(), + "importance": importance, + } + (output_dir / "metrics.json").write_text(json.dumps(payload, indent=2), encoding="utf-8") + + use_wandb = wandb_is_configured() + if use_wandb: + import os + + import wandb + + ensure_wandb_project() + wandb.init( + entity=WANDB_ENTITY, + project=WANDB_PROJECT, + name=os.environ.get("WANDB_NAME", model_name), + config={ + "model": model_name, + "method": method, + "n_estimators": n_estimators, + "max_depth": max_depth, + "C": C, + "n_features": len(names), + }, + ) + log_action_charts_to_wandb( + metrics, + y_true=y_val, + y_pred=y_pred, + importance=importance, + artifact_dir=report_dir, + prefix="eval", + ) + wandb.finish() + else: + from pokerai.eval.visualize import ( + save_confusion_png, + save_importance_png, + save_per_class_f1_png, + ) + + save_confusion_png(metrics, report_dir / "confusion.png") + save_per_class_f1_png(metrics, report_dir / "per_class_f1.png") + if importance: + save_importance_png(importance, report_dir / "feature_importance.png") + + print(f"Saved to {output_dir}") + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/training/train_gpt2.py b/src/pokerai/training/train_gpt2.py new file mode 100644 index 0000000..7a27e31 --- /dev/null +++ b/src/pokerai/training/train_gpt2.py @@ -0,0 +1,265 @@ +"""Train GPT-2 from scratch with a raw PyTorch loop (action-only loss).""" + +from __future__ import annotations + +import os +from collections import Counter +from pathlib import Path +from typing import cast + +import torch +import torch.nn as nn +import torch.nn.functional as F +from torch.utils.data import DataLoader, Dataset + +from pokerai.config import ( + GPT2Hyperparams, + HANDS_CLEAN, + MODEL_GPT2_DIR, + WANDB_ENTITY, + WANDB_PROJECT, +) +from pokerai.data import encode_with_mask, load_text_split, split_prompt_completion +from pokerai.eval.actions import action_type +from pokerai.models import build_gpt2 +from pokerai.training import ( + ensure_wandb_project, + get_device, + load_tokenizer, + wandb_is_configured, +) + + +class HandsDataset(Dataset): + def __init__( + self, + examples: list[tuple[list[int], list[int]]], + weights: list[float] | None = None, + ): + self.examples = examples + self.weights = weights or [1.0] * len(examples) + + def __len__(self) -> int: + return len(self.examples) + + def __getitem__(self, idx: int): + ids, labels = self.examples[idx] + return ids, labels, self.weights[idx] + + +def make_collate(pad_id: int): + def collate(batch): + max_len = max(len(ids) for ids, _, _ in batch) + input_ids, labels, attn_mask, weights = [], [], [], [] + for ids, lbls, w in batch: + pad_len = max_len - len(ids) + input_ids.append(ids + [pad_id] * pad_len) + labels.append(lbls + [-100] * pad_len) + attn_mask.append([1] * len(ids) + [0] * pad_len) + weights.append(w) + return ( + torch.tensor(input_ids), + torch.tensor(labels), + torch.tensor(attn_mask), + torch.tensor(weights, dtype=torch.float), + ) + + return collate + + +def _inverse_freq_weights(texts: list[str]) -> list[float]: + types = [action_type(split_prompt_completion(t)[1]) for t in texts] + counts = Counter(types) + n = len(types) + # weight_c = n / (K * count_c) + k = max(1, len([c for c in counts.values() if c > 0])) + return [n / (k * counts[t]) for t in types] + + +def _token_loss(logits, targets, vocab_size: int, weights=None) -> torch.Tensor: + """Mean CE over non-ignored tokens; optional per-sequence weights.""" + flat_logits = logits.reshape(-1, vocab_size) + flat_targets = targets.reshape(-1) + per_tok = F.cross_entropy( + flat_logits, flat_targets, ignore_index=-100, reduction="none" + ) + per_tok = per_tok.view(targets.shape) + mask = targets != -100 + if weights is None: + return per_tok[mask].mean() if mask.any() else per_tok.sum() * 0.0 + # Mean over tokens within each sequence, then weighted mean over batch. + tok_counts = mask.sum(dim=1).clamp(min=1).float() + seq_loss = (per_tok * mask.float()).sum(dim=1) / tok_counts + w = weights.to(seq_loss.device) + return (seq_loss * w).sum() / w.sum().clamp(min=1e-8) + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_GPT2_DIR, + hp: GPT2Hyperparams | None = None, + *, + class_weight: bool = False, + run_action_eval: bool = True, + eval_max_examples: int = 256, +) -> None: + hp = hp or GPT2Hyperparams() + tokenizer = load_tokenizer() + vocab_size = len(tokenizer) + if tokenizer.pad_token_id is None: + raise ValueError("Tokenizer is missing pad_token_id") + model = build_gpt2(tokenizer, hp) + n_params = sum(p.numel() for p in model.parameters()) + print(f"Model has {n_params:,} parameters") + if class_weight: + print("Using inverse-frequency action-type class weights") + + device = get_device() + cast(nn.Module, model).to(device) + print("Training on:", device) + + split = load_text_split(hands_path) + train_texts = [ex["text"] for ex in split["train"]] + val_texts = [ex["text"] for ex in split["test"]] + train_enc = [encode_with_mask(t, tokenizer, hp.n_positions) for t in train_texts] + val_enc = [encode_with_mask(t, tokenizer, hp.n_positions) for t in val_texts] + train_w = _inverse_freq_weights(train_texts) if class_weight else None + train_ds = HandsDataset(train_enc, train_w) + val_ds = HandsDataset(val_enc) + collate = make_collate(tokenizer.pad_token_id) + train_loader = DataLoader( + train_ds, batch_size=hp.batch_size, shuffle=True, collate_fn=collate + ) + val_loader = DataLoader( + val_ds, batch_size=hp.batch_size, shuffle=False, collate_fn=collate + ) + + use_wandb = wandb_is_configured() + run_name = os.environ.get( + "WANDB_NAME", "gpt2-weighted" if class_weight else "gpt2-raw" + ) + if use_wandb: + import wandb + + ensure_wandb_project() + wandb.init( + entity=WANDB_ENTITY, + project=WANDB_PROJECT, + name=run_name, + config={ + "model": "gpt2-weighted" if class_weight else "gpt2", + "class_weight": class_weight, + "n_params": n_params, + "vocab_size": vocab_size, + "device": device, + "train_examples": len(train_ds), + "eval_examples": len(val_ds), + **hp.as_dict(), + }, + ) + + ids, labels, _w = train_ds[0] + print("\nTokens:", tokenizer.convert_ids_to_tokens(ids)) + print( + "Labels:", + [ + tokenizer.convert_ids_to_tokens([t])[0] if t != -100 else "---" + for t in labels + ], + ) + print("(Only non-'---' positions contribute to the loss.)\n") + + optimizer = torch.optim.AdamW(model.parameters(), lr=hp.learning_rate) + + def run_eval() -> float: + model.eval() + total_loss, total_tokens = 0.0, 0 + with torch.no_grad(): + for input_ids, batch_labels, attn_mask, _weights in val_loader: + input_ids = input_ids.to(device) + batch_labels = batch_labels.to(device) + attn_mask = attn_mask.to(device) + logits = model(input_ids=input_ids, attention_mask=attn_mask).logits[ + :, :-1, : + ].contiguous() + targets = batch_labels[:, 1:].contiguous() + loss = F.cross_entropy( + logits.reshape(-1, vocab_size), + targets.reshape(-1), + ignore_index=-100, + ) + n_tok = (targets != -100).sum().item() + total_loss += loss.item() * n_tok + total_tokens += n_tok + model.train() + return total_loss / max(total_tokens, 1) + + step = 0 + val_loss = float("nan") + for epoch in range(hp.num_epochs): + for input_ids, batch_labels, attn_mask, weights in train_loader: + input_ids = input_ids.to(device) + batch_labels = batch_labels.to(device) + attn_mask = attn_mask.to(device) + weights = weights.to(device) + logits = model(input_ids=input_ids, attention_mask=attn_mask).logits[ + :, :-1, : + ].contiguous() + targets = batch_labels[:, 1:].contiguous() + loss = _token_loss( + logits, + targets, + vocab_size, + weights=weights if class_weight else None, + ) + optimizer.zero_grad() + loss.backward() + optimizer.step() + + if step % 50 == 0: + print(f"epoch {epoch} step {step:5d} | train loss {loss.item():.4f}") + if use_wandb: + import wandb + + wandb.log({"train/loss": loss.item(), "epoch": epoch}, step=step) + step += 1 + + val_loss = run_eval() + print(f"== end of epoch {epoch}: val loss {val_loss:.4f} ==") + if use_wandb: + import wandb + + wandb.log({"val/loss": val_loss, "epoch": epoch}, step=step) + + output_dir.mkdir(parents=True, exist_ok=True) + model.save_pretrained(str(output_dir)) + tokenizer.save_pretrained(str(output_dir)) + print(f"\nSaved model to {output_dir}") + + if run_action_eval: + from pokerai.eval.report import run_lm_evaluation_report + + report_dir = Path(output_dir) / "report" + print(f"Running action-type evaluation (max {eval_max_examples} examples)...") + summary = run_lm_evaluation_report( + model, + tokenizer, + val_texts, + report_dir=report_dir, + max_examples=eval_max_examples, + log_wandb=use_wandb, + ) + m = summary["metrics"] + print( + f"Action eval: accuracy={m['accuracy']:.4f} macro_f1={m['macro_f1']:.4f}" + ) + + if use_wandb: + import wandb + + wandb.summary["final_val_loss"] = val_loss + wandb.finish() + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/training/train_majority.py b/src/pokerai/training/train_majority.py new file mode 100644 index 0000000..0667ce0 --- /dev/null +++ b/src/pokerai/training/train_majority.py @@ -0,0 +1,88 @@ +"""Majority-class action-type baseline (imbalance floor).""" + +from __future__ import annotations + +import json +from collections import Counter +from pathlib import Path + +from pokerai.config import ARTIFACTS_DIR, HANDS_CLEAN, WANDB_ENTITY, WANDB_PROJECT +from pokerai.data import load_text_split, split_prompt_completion +from pokerai.eval.actions import action_type, action_type_counts +from pokerai.eval.metrics import compute_action_metrics +from pokerai.eval.visualize import log_action_charts_to_wandb +from pokerai.training import ensure_wandb_project, wandb_is_configured + +MODEL_MAJORITY_DIR = ARTIFACTS_DIR / "models" / "majority" + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_MAJORITY_DIR, +) -> None: + split = load_text_split(hands_path) + train_actions = [ + action_type(split_prompt_completion(ex["text"])[1]) for ex in split["train"] + ] + val_true = [ + action_type(split_prompt_completion(ex["text"])[1]) for ex in split["test"] + ] + majority = Counter(train_actions).most_common(1)[0][0] + val_pred = [majority] * len(val_true) + metrics = compute_action_metrics(val_true, val_pred) + dist = action_type_counts(train_actions) + + print(f"Majority class: {majority}") + print(f"Train distribution: {dist}") + print( + f"Val accuracy={metrics.accuracy:.4f} macro_f1={metrics.macro_f1:.4f} n={metrics.n}" + ) + for name, scores in metrics.per_class.items(): + print( + f" {name:6s} P={scores.precision:.3f} R={scores.recall:.3f} " + f"F1={scores.f1:.3f} support={scores.support}" + ) + + output_dir.mkdir(parents=True, exist_ok=True) + report_dir = output_dir / "report" + payload = { + "model": "majority", + "majority_class": majority, + "train_distribution": dist, + "metrics": metrics.as_dict(), + } + (output_dir / "majority.json").write_text(json.dumps(payload, indent=2), encoding="utf-8") + + use_wandb = wandb_is_configured() + if use_wandb: + import os + + import wandb + + ensure_wandb_project() + wandb.init( + entity=WANDB_ENTITY, + project=WANDB_PROJECT, + name=os.environ.get("WANDB_NAME", "majority-baseline"), + config={"model": "majority", "majority_class": majority, **dist}, + ) + log_action_charts_to_wandb( + metrics, + y_true=val_true, + y_pred=val_pred, + artifact_dir=report_dir, + prefix="eval", + ) + wandb.summary["majority_class"] = majority + wandb.finish() + else: + from pokerai.eval.visualize import save_confusion_png, save_per_class_f1_png + + save_confusion_png(metrics, report_dir / "confusion.png", title="Majority confusion") + save_per_class_f1_png(metrics, report_dir / "per_class_f1.png") + + print(f"Saved to {output_dir}") + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/training/train_trl.py b/src/pokerai/training/train_trl.py new file mode 100644 index 0000000..f46b9ca --- /dev/null +++ b/src/pokerai/training/train_trl.py @@ -0,0 +1,93 @@ +"""Train GPT-2 via Hugging Face TRL SFTTrainer (completion-only loss).""" + +from __future__ import annotations + +import os +from pathlib import Path + +import torch +from trl import SFTConfig, SFTTrainer + +from pokerai.config import ( + GPT2Hyperparams, + HANDS_CLEAN, + MODEL_TRL_DIR, +) +from pokerai.data import encode_with_mask, load_text_split +from pokerai.models import build_gpt2 +from pokerai.training import ( + ensure_wandb_project, + get_device, + load_tokenizer, + wandb_report_to, +) + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_TRL_DIR, + hp: GPT2Hyperparams | None = None, +) -> None: + hp = hp or GPT2Hyperparams() + tokenizer = load_tokenizer() + model = build_gpt2(tokenizer, hp) + n_params = sum(p.numel() for p in model.parameters()) + print(f"Model has {n_params:,} parameters") + device = get_device() + print("Device preference:", device) + + # Pre-tokenize with the same encode_with_mask path as the raw GPT-2 loop: + # BOS/EOS, offset-based action mask, keep-end truncation. TRL's built-in + # truncation_mode="keep_end" is deprecated and defaults to keep_start, + # which can drop the action tokens entirely on long hands. + def _tokenize(example: dict) -> dict: + ids, labels = encode_with_mask(example["text"], tokenizer, hp.n_positions) + return {"input_ids": ids, "labels": labels} + + split = load_text_split(hands_path) + train_dataset = split["train"].map(_tokenize, remove_columns=["text"]) + eval_dataset = split["test"].map(_tokenize, remove_columns=["text"]) + print("Example 0 input_ids length:", len(train_dataset[0]["input_ids"])) + print( + "Example 0 supervised tokens:", + sum(1 for t in train_dataset[0]["labels"] if t != -100), + ) + + use_cuda = torch.cuda.is_available() + # TRL defaults bf16 = not fp16 when bf16 is None, which crashes on CPU. + # Let HF Trainer own W&B init via report_to — avoid a second wandb.init(). + training_args = SFTConfig( + output_dir=str(output_dir), + num_train_epochs=hp.num_epochs, + per_device_train_batch_size=hp.batch_size, + per_device_eval_batch_size=hp.batch_size, + eval_strategy="epoch", + logging_steps=50, + learning_rate=hp.learning_rate, + # Dataset already truncated + masked; skip TRL's prepare/tokenize. + max_length=None, + dataset_kwargs={"skip_prepare_dataset": True}, + fp16=use_cuda, + bf16=False, + use_cpu=not use_cuda, + report_to=wandb_report_to(), + run_name=os.environ.get("WANDB_NAME", "gpt2-trl"), + ) + + ensure_wandb_project() + + trainer = SFTTrainer( + model=model, + args=training_args, + train_dataset=train_dataset, + eval_dataset=eval_dataset, + processing_class=tokenizer, + ) + trainer.train() + trainer.save_model(str(output_dir)) + tokenizer.save_pretrained(str(output_dir)) + print(f"Saved to {output_dir}/") + + +if __name__ == "__main__": + main() diff --git a/src/pokerai/training/train_weighted_gpt2.py b/src/pokerai/training/train_weighted_gpt2.py new file mode 100644 index 0000000..6610feb --- /dev/null +++ b/src/pokerai/training/train_weighted_gpt2.py @@ -0,0 +1,28 @@ +"""GPT-2 with inverse-frequency action-type class weights (Family F3).""" + +from __future__ import annotations + +from pathlib import Path + +from pokerai.config import ARTIFACTS_DIR, GPT2Hyperparams, HANDS_CLEAN +from pokerai.training.train_gpt2 import main as train_gpt2_main + +MODEL_WEIGHTED_DIR = ARTIFACTS_DIR / "models" / "gpt2_weighted" + + +def main( + hands_path: Path = HANDS_CLEAN, + output_dir: Path = MODEL_WEIGHTED_DIR, + hp: GPT2Hyperparams | None = None, +) -> None: + train_gpt2_main( + hands_path=hands_path, + output_dir=output_dir, + hp=hp, + class_weight=True, + run_action_eval=True, + ) + + +if __name__ == "__main__": + main() diff --git a/tests/test_config.py b/tests/test_config.py new file mode 100644 index 0000000..323dcd3 --- /dev/null +++ b/tests/test_config.py @@ -0,0 +1,54 @@ +"""Config and training-helper sanity checks.""" + +import os + +from pokerai.config import ( + BOS_TOKEN, + EOS_TOKEN, + N_POSITIONS, + PAD_TOKEN, + REPO_ROOT, + WANDB_ENTITY, + WANDB_PROJECT, +) +from pokerai.training import ensure_wandb_project, wandb_is_configured, wandb_report_to + + +def test_special_tokens_are_distinct(): + assert BOS_TOKEN != EOS_TOKEN + assert PAD_TOKEN not in (BOS_TOKEN, EOS_TOKEN) + + +def test_context_length(): + assert N_POSITIONS >= 384 + + +def test_repo_root_exists(): + assert (REPO_ROOT / "README.md").exists() + assert (REPO_ROOT / "src" / "pokerai").is_dir() + + +def test_wandb_team_project(): + assert WANDB_ENTITY == "mooslin-university-of-wisconsin-madison" + assert WANDB_PROJECT == "poker-ai" + + +def test_ensure_wandb_project_sets_env(monkeypatch): + monkeypatch.delenv("WANDB_ENTITY", raising=False) + monkeypatch.delenv("WANDB_PROJECT", raising=False) + ensure_wandb_project() + assert os.environ["WANDB_ENTITY"] == WANDB_ENTITY + assert os.environ["WANDB_PROJECT"] == WANDB_PROJECT + + +def test_wandb_disabled_by_env(monkeypatch): + monkeypatch.setenv("WANDB_MODE", "disabled") + monkeypatch.delenv("WANDB_API_KEY", raising=False) + assert wandb_is_configured() is False + assert wandb_report_to() == "none" + + +def test_wandb_offline_mode(monkeypatch): + monkeypatch.setenv("WANDB_MODE", "offline") + assert wandb_is_configured() is True + assert wandb_report_to() == "wandb" diff --git a/tests/test_data.py b/tests/test_data.py new file mode 100644 index 0000000..c837118 --- /dev/null +++ b/tests/test_data.py @@ -0,0 +1,149 @@ +"""Unit tests for data cleaning and action-mask encoding.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from tokenizers import ByteLevelBPETokenizer +from transformers import PreTrainedTokenizerFast + +from pokerai.data import ( + ACTION_SEP, + clean_line, + encode_with_mask, + load_text_split, + split_prompt_completion, + to_line, +) + + +class _FakeTok: + """Minimal tokenizer stub: one char -> one id, specials at high ids.""" + + bos_token_id = 100 + eos_token_id = 101 + pad_token_id = 102 + + def __call__(self, text, add_special_tokens=False, return_offsets_mapping=False): + ids = [ord(c) % 97 for c in text] + out = {"input_ids": ids} + if return_offsets_mapping: + out["offset_mapping"] = [(i, i + 1) for i in range(len(text))] + return out + + +def _train_tiny_bpe(tmpdir: Path, lines: list[str]) -> PreTrainedTokenizerFast: + corpus = tmpdir / "hands.txt" + corpus.write_text("\n".join(lines) + "\n", encoding="utf-8") + bpe = ByteLevelBPETokenizer() + bpe.train( + files=[str(corpus)], + vocab_size=200, + min_frequency=1, + special_tokens=["<|startoftext|>", "<|endoftext|>", "<|pad|>"], + ) + tok_json = tmpdir / "tokenizer.json" + bpe.save(str(tok_json)) + tok = PreTrainedTokenizerFast(tokenizer_file=str(tok_json)) + tok.add_special_tokens( + { + "pad_token": "<|pad|>", + "bos_token": "<|startoftext|>", + "eos_token": "<|endoftext|>", + } + ) + return tok + + +def test_to_line_flattens_whitespace(): + assert to_line("a\nb c", "FOLD") == "a b c,FOLD" + + +def test_action_sep_is_comma(): + assert ACTION_SEP == "," + + +def test_clean_call_zero_bb(): + line = "[PREFLOP] P1:,CALL 0BB" + assert clean_line(line) == "[PREFLOP] P1:,CALL" + + +def test_clean_fold_with_amount(): + assert clean_line("state,FOLD0BB") == "state,FOLD" + assert clean_line("state,FOLD") == "state,FOLD" + + +def test_split_prompt_completion(): + prompt, completion = split_prompt_completion("ctx,a,b,FOLD") + assert prompt == "ctx,a,b," + assert completion == "FOLD" + + +def test_split_requires_comma(): + with pytest.raises(ValueError): + split_prompt_completion("no-comma-here") + + +def test_split_rejects_arrow_separator(): + """Legacy ' => ' format must not be treated as a valid hand line.""" + with pytest.raises(ValueError): + split_prompt_completion("state => FOLD") + + +def test_encode_with_mask_supervises_action_only(): + tok = _FakeTok() + text = "ABC,XY" + ids, labels = encode_with_mask(text, tok, max_length=64) + assert ids[0] == tok.bos_token_id + assert ids[-1] == tok.eos_token_id + # Prompt "ABC," -> 4 tokens after BOS are masked; "XY" + EOS supervised + prompt_len = 1 + len(tok("ABC,", add_special_tokens=False)["input_ids"]) + assert all(t == -100 for t in labels[:prompt_len]) + assert all(t != -100 for t in labels[prompt_len:]) + assert labels[prompt_len:] == ids[prompt_len:] + + +def test_encode_truncates_from_front(): + tok = _FakeTok() + text = "ABCDEFGHIJ,Z" + ids, labels = encode_with_mask(text, tok, max_length=6) + assert len(ids) == 6 + assert ids[-1] == tok.eos_token_id + # Action region should still be present at the end + assert labels[-1] == tok.eos_token_id + + +def test_encode_with_real_bpe_matches_offsets(tmp_path: Path): + lines = [ + "[PREFLOP] P3: RAISE 2BB P1:,FOLD", + "[STACKS] P1: 44.2BB [Qh 9h] P2: 103.4BB,CALL", + "BTN SB BB POT=1.5BB [PREFLOP] P3: RAISE 2BB P1:,RAISE 6BB", + ] * 10 + tok = _train_tiny_bpe(tmp_path, lines) + text = lines[0] + ids, labels = encode_with_mask(text, tok, max_length=128) + cut = text.rfind(",") + 1 + enc = tok(text, add_special_tokens=False, return_offsets_mapping=True) + prompt_body = next( + (i for i, (s, _) in enumerate(enc["offset_mapping"]) if s >= cut), + len(enc["input_ids"]), + ) + prompt_len = 1 + prompt_body + assert ids == [tok.bos_token_id] + list(enc["input_ids"]) + [tok.eos_token_id] + assert all(t == -100 for t in labels[:prompt_len]) + assert labels[prompt_len:] == ids[prompt_len:] + + +def test_load_text_split_rejects_empty_file(tmp_path: Path): + empty = tmp_path / "empty.txt" + empty.write_text("", encoding="utf-8") + with pytest.raises(ValueError, match="empty"): + load_text_split(empty) + + +def test_load_text_split_skips_blank_lines(tmp_path: Path): + path = tmp_path / "hands.txt" + path.write_text("a,FOLD\n\n\nb,CALL\n", encoding="utf-8") + split = load_text_split(path, test_size=0.5, seed=0) + assert len(split["train"]) + len(split["test"]) == 2 diff --git a/tests/test_eval.py b/tests/test_eval.py new file mode 100644 index 0000000..fd8ec14 --- /dev/null +++ b/tests/test_eval.py @@ -0,0 +1,96 @@ +"""Tests for action metrics, features, and occlusion importance.""" + +from __future__ import annotations + +from pathlib import Path + +from pokerai.eval.actions import action_type, action_type_counts +from pokerai.eval.importance import occlude_region, occlusion_importance +from pokerai.eval.metrics import compute_action_metrics +from pokerai.eval.visualize import save_confusion_png, save_per_class_f1_png +from pokerai.features import extract_features, feature_names, features_and_label + + +def test_action_type_parsing(): + assert action_type("FOLD") == "FOLD" + assert action_type("CALL 2BB") == "CALL" + assert action_type("RAISE 6.5BB") == "RAISE" + assert action_type("CHECK") == "CHECK" + assert action_type("BET 3BB") == "BET" + assert action_type("ALL-IN") == "RAISE" + assert action_type("???") == "OTHER" + + +def test_compute_action_metrics_perfect(): + y = ["FOLD", "CALL", "RAISE 2BB", "CHECK"] + m = compute_action_metrics(y, y) + assert m.accuracy == 1.0 + assert m.macro_f1 == 1.0 + assert m.per_class["FOLD"].support == 1 + + +def test_compute_action_metrics_confusion_shape(): + m = compute_action_metrics(["FOLD", "CALL"], ["CALL", "CALL"]) + assert len(m.confusion) == 6 + assert m.accuracy == 0.5 + flat = m.flat_wandb_metrics() + assert "action/call/f1" in flat + + +def test_extract_features_basic(): + prompt = ( + "[TABLE_CONFIGURATION] BTN=P3 SB=P1 0.5BB BB=P2 1BB " + "[STACKS] P1: 44.2BB [Qh 9h] P2: 103.4BB P3: 165.2BB POT=1.5BB " + "[PREFLOP] P3: RAISE 2BB P1:" + ) + feats = extract_features(prompt) + assert feats.pot_bb == 1.5 + assert feats.n_stacks == 3 + assert feats.street_preflop == 1 + assert feats.n_raises >= 1 + assert feats.has_hole_cards == 1 + assert feats.hole_suited == 1 + assert len(feats.vector()) == len(feature_names()) + + +def test_features_and_label(): + line = "[STACKS] P1: 10BB [Ah Ad] POT=1.5BB [PREFLOP] P1:,RAISE 3BB" + vec, label = features_and_label(line) + assert label == "RAISE" + assert len(vec) == len(feature_names()) + + +def test_occlusion_changes_text(): + prompt = "[STACKS] P1: 10BB [Ah Kd] POT=2BB [PREFLOP] P1:" + masked = occlude_region(prompt, "pot") + assert "POT=" not in masked + assert "POT_MASKED" in masked or "pot".upper() in masked.upper() + + +def test_occlusion_importance_ordering(): + prompts = ["AAAA POT=1BB", "BBBB POT=2BB", "CCCC POT=3BB"] + + def score(ps: list[str]) -> float: + # Higher when POT= is present. + return sum(1.0 for p in ps if "POT=" in p) / len(ps) + + scores = occlusion_importance(prompts, score, regions=["pot", "stacks"]) + assert scores["pot"] > scores["stacks"] + + +def test_save_charts(tmp_path: Path): + m = compute_action_metrics( + ["FOLD", "CALL", "RAISE", "FOLD"], + ["FOLD", "FOLD", "RAISE", "CALL"], + ) + cm = save_confusion_png(m, tmp_path / "cm.png") + f1 = save_per_class_f1_png(m, tmp_path / "f1.png") + assert cm.exists() and cm.stat().st_size > 0 + assert f1.exists() and f1.stat().st_size > 0 + + +def test_action_type_counts(): + counts = action_type_counts(["FOLD", "FOLD", "CALL 1BB"]) + assert counts["FOLD"] == 2 + assert counts["CALL"] == 1 + assert counts["RAISE"] == 0 diff --git a/tests/test_run_experiment.py b/tests/test_run_experiment.py new file mode 100644 index 0000000..13341a0 --- /dev/null +++ b/tests/test_run_experiment.py @@ -0,0 +1,41 @@ +"""Tests for the experiment launcher W&B gating.""" + +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parents[1] +LAUNCHER = REPO / "scripts" / "run_experiment.py" + + +def test_require_wandb_fails_when_disabled(): + env = os.environ.copy() + env["WANDB_MODE"] = "disabled" + env.pop("WANDB_API_KEY", None) + proc = subprocess.run( + [sys.executable, str(LAUNCHER), "bigram", "--require-wandb"], + cwd=REPO, + capture_output=True, + text=True, + env=env, + ) + assert proc.returncode != 0 + combined = proc.stderr + proc.stdout + assert "not configured" in combined or "--require-wandb" in combined + + +def test_launcher_help_mentions_optional_wandb(): + proc = subprocess.run( + [sys.executable, str(LAUNCHER), "--help"], + cwd=REPO, + capture_output=True, + text=True, + ) + assert proc.returncode == 0 + assert "--require-wandb" in proc.stdout + assert "optional" in proc.stdout.lower() + for trainer in ("majority", "features", "weighted-gpt2", "gpt2", "bigram", "trl"): + assert trainer in proc.stdout diff --git a/tests/test_trl_config.py b/tests/test_trl_config.py new file mode 100644 index 0000000..554f44e --- /dev/null +++ b/tests/test_trl_config.py @@ -0,0 +1,29 @@ +"""Regression tests for TRL training-arg safety on CPU.""" + +from __future__ import annotations + +import torch +from trl import SFTConfig + +from pokerai.training import wandb_report_to + + +def test_sft_config_cpu_safe(monkeypatch): + """TRL sets bf16=not fp16 when bf16 is None; we must set bf16=False on CPU.""" + monkeypatch.setenv("WANDB_MODE", "disabled") + use_cuda = torch.cuda.is_available() + cfg = SFTConfig( + output_dir="/tmp/pokerai-trl-test", + num_train_epochs=1, + per_device_train_batch_size=1, + max_length=None, + dataset_kwargs={"skip_prepare_dataset": True}, + fp16=use_cuda, + bf16=False, + use_cpu=not use_cuda, + report_to=wandb_report_to(), + ) + assert cfg.bf16 is False + if not use_cuda: + assert cfg.fp16 is False + assert cfg.dataset_kwargs["skip_prepare_dataset"] is True diff --git a/tokenizer/tokenizer.json b/tokenizer/tokenizer.json new file mode 100644 index 0000000..49ad49a --- /dev/null +++ b/tokenizer/tokenizer.json @@ -0,0 +1,3506 @@ +{ + "version": "1.0", + "truncation": null, + "padding": null, + "added_tokens": [ + { + "id": 0, + "content": "<|startoftext|>", + "single_word": false, + "lstrip": false, + "rstrip": false, + "normalized": false, + "special": true + }, + { + "id": 1, + "content": "<|pad|>", + "single_word": false, + "lstrip": false, + "rstrip": false, + "normalized": false, + "special": true + } + ], + "normalizer": null, + "pre_tokenizer": { + "type": "ByteLevel", + "add_prefix_space": false, + "trim_offsets": true, + "use_regex": true + }, + "post_processor": { + "type": "ByteLevel", + "add_prefix_space": true, + "trim_offsets": false, + "use_regex": true + }, + "decoder": { + "type": "ByteLevel", + "add_prefix_space": true, + "trim_offsets": true, + "use_regex": true + }, + "model": { + "type": "BPE", + "dropout": null, + "unk_token": null, + "continuing_subword_prefix": null, + "end_of_word_suffix": null, + "fuse_unk": false, + "byte_fallback": false, + "ignore_merges": false, + "vocab": { + "<|startoftext|>": 0, + "<|pad|>": 1, + "!": 2, + "\"": 3, + "#": 4, + "$": 5, + "%": 6, + "&": 7, + "'": 8, + "(": 9, + ")": 10, + "*": 11, + "+": 12, + ",": 13, + "-": 14, + ".": 15, + "/": 16, + "0": 17, + "1": 18, + "2": 19, + "3": 20, + "4": 21, + "5": 22, + "6": 23, + "7": 24, + "8": 25, + "9": 26, + ":": 27, + ";": 28, + "<": 29, + "=": 30, + ">": 31, + "?": 32, + "@": 33, + "A": 34, + "B": 35, + "C": 36, + "D": 37, + "E": 38, + "F": 39, + "G": 40, + "H": 41, + "I": 42, + "J": 43, + "K": 44, + "L": 45, + "M": 46, + "N": 47, + "O": 48, + "P": 49, + "Q": 50, + "R": 51, + "S": 52, + "T": 53, + "U": 54, + "V": 55, + "W": 56, + "X": 57, + "Y": 58, + "Z": 59, + "[": 60, + "\\": 61, + "]": 62, + "^": 63, + "_": 64, + "`": 65, + "a": 66, + "b": 67, + "c": 68, + "d": 69, + "e": 70, + "f": 71, + "g": 72, + "h": 73, + "i": 74, + "j": 75, + "k": 76, + "l": 77, + "m": 78, + "n": 79, + "o": 80, + "p": 81, + "q": 82, + "r": 83, + "s": 84, + "t": 85, + "u": 86, + "v": 87, + "w": 88, + "x": 89, + "y": 90, + "z": 91, + "{": 92, + "|": 93, + "}": 94, + "~": 95, + "¡": 96, + "¢": 97, + "£": 98, + "¤": 99, + "¥": 100, + "¦": 101, + "§": 102, + "¨": 103, + "©": 104, + "ª": 105, + "«": 106, + "¬": 107, + "®": 108, + "¯": 109, + "°": 110, + "±": 111, + "²": 112, + "³": 113, + "´": 114, + "µ": 115, + "¶": 116, + "·": 117, + "¸": 118, + "¹": 119, + "º": 120, + "»": 121, + "¼": 122, + "½": 123, + "¾": 124, + "¿": 125, + "À": 126, + "Á": 127, + "Â": 128, + "Ã": 129, + "Ä": 130, + "Å": 131, + "Æ": 132, + "Ç": 133, + "È": 134, + "É": 135, + "Ê": 136, + "Ë": 137, + "Ì": 138, + "Í": 139, + "Î": 140, + "Ï": 141, + "Ð": 142, + "Ñ": 143, + "Ò": 144, + "Ó": 145, + "Ô": 146, + "Õ": 147, + "Ö": 148, + "×": 149, + "Ø": 150, + "Ù": 151, + "Ú": 152, + "Û": 153, + "Ü": 154, + "Ý": 155, + "Þ": 156, + "ß": 157, + "à": 158, + "á": 159, + "â": 160, + "ã": 161, + "ä": 162, + "å": 163, + "æ": 164, + "ç": 165, + "è": 166, + "é": 167, + "ê": 168, + "ë": 169, + "ì": 170, + "í": 171, + "î": 172, + "ï": 173, + "ð": 174, + "ñ": 175, + "ò": 176, + "ó": 177, + "ô": 178, + "õ": 179, + "ö": 180, + "÷": 181, + "ø": 182, + "ù": 183, + "ú": 184, + "û": 185, + "ü": 186, + "ý": 187, + "þ": 188, + "ÿ": 189, + "Ā": 190, + "ā": 191, + "Ă": 192, + "ă": 193, + "Ą": 194, + "ą": 195, + "Ć": 196, + "ć": 197, + "Ĉ": 198, + "ĉ": 199, + "Ċ": 200, + "ċ": 201, + "Č": 202, + "č": 203, + "Ď": 204, + "ď": 205, + "Đ": 206, + "đ": 207, + "Ē": 208, + "ē": 209, + "Ĕ": 210, + "ĕ": 211, + "Ė": 212, + "ė": 213, + "Ę": 214, + "ę": 215, + "Ě": 216, + "ě": 217, + "Ĝ": 218, + "ĝ": 219, + "Ğ": 220, + "ğ": 221, + "Ġ": 222, + "ġ": 223, + "Ģ": 224, + "ģ": 225, + "Ĥ": 226, + "ĥ": 227, + "Ħ": 228, + "ħ": 229, + "Ĩ": 230, + "ĩ": 231, + "Ī": 232, + "ī": 233, + "Ĭ": 234, + "ĭ": 235, + "Į": 236, + "į": 237, + "İ": 238, + "ı": 239, + "IJ": 240, + "ij": 241, + "Ĵ": 242, + "ĵ": 243, + "Ķ": 244, + "ķ": 245, + "ĸ": 246, + "Ĺ": 247, + "ĺ": 248, + "Ļ": 249, + "ļ": 250, + "Ľ": 251, + "ľ": 252, + "Ŀ": 253, + "ŀ": 254, + "Ł": 255, + "ł": 256, + "Ń": 257, + "ĠP": 258, + "BB": 259, + "Ġ1": 260, + "Ġ[": 261, + "FO": 262, + "LD": 263, + "FOLD": 264, + "TA": 265, + "ĠFOLD": 266, + "ON": 267, + "CK": 268, + "RA": 269, + "Ġ10": 270, + "OT": 271, + "STA": 272, + "ĠPOT": 273, + "CKS": 274, + "STACKS": 275, + "Ġ3": 276, + "FL": 277, + "OP": 278, + "FLOP": 279, + "Ġ2": 280, + "ĠB": 281, + "Ġ4": 282, + "Ġ0": 283, + ":,": 284, + "BL": 285, + "CON": 286, + "EFLOP": 287, + "FI": 288, + "GU": 289, + "ION": 290, + "PR": 291, + "SB": 292, + "TN": 293, + "TION": 294, + "ĠBB": 295, + "ĠSB": 296, + "TABL": 297, + "RATION": 298, + "ĠBTN": 299, + "CONFI": 300, + "GURATION": 301, + "PREFLOP": 302, + "TABLE": 303, + "CONFIGURATION": 304, + "ĠC": 305, + "IS": 306, + "RAIS": 307, + "RAISE": 308, + "Ġ5": 309, + "Ġ9": 310, + "ĠRAISE": 311, + "AL": 312, + "ALL": 313, + "Ġ100": 314, + "ĠCALL": 315, + "ECK": 316, + "HECK": 317, + "Ġ6": 318, + "Ġ11": 319, + "Ġ7": 320, + "][": 321, + "Ġ8": 322, + "ĠCHECK": 323, + "Ġ39": 324, + "Ġ12": 325, + "ĠA": 326, + "Ġ13": 327, + "ET": 328, + "ĠK": 329, + "ĠQ": 330, + "Ġ40": 331, + "Ġ14": 332, + "ĠJ": 333, + "10": 334, + "Ġ101": 335, + "Ġ99": 336, + "ĠBET": 337, + "Ġ19": 338, + "Ġ15": 339, + "Ġ38": 340, + "Ġ50": 341, + "Ġ16": 342, + "RN": 343, + "TU": 344, + "TURN": 345, + "CHECK": 346, + "Ġ18": 347, + "Ġ17": 348, + "Ġ103": 349, + "Ġ20": 350, + "Ġ102": 351, + "Ġ41": 352, + "Ġ104": 353, + "Ġ42": 354, + "Ġ43": 355, + "Ġ37": 356, + "Ġ49": 357, + "Ġ105": 358, + "CALL": 359, + "Ġ21": 360, + "Ġ106": 361, + "Ġ44": 362, + "Ġ22": 363, + "Ġ98": 364, + "Ġ107": 365, + "ĠAc": 366, + "Ġ51": 367, + "BET": 368, + "Ġ108": 369, + "ĠAd": 370, + "Ġ36": 371, + "ĠAh": 372, + "Ġ23": 373, + "ĠKh": 374, + "Ġ109": 375, + "Ġ45": 376, + "Ġ48": 377, + "ĠAs": 378, + "Ah": 379, + "Ġ110": 380, + "ĠKc": 381, + "Ġ111": 382, + "Ġ47": 383, + "ĠQd": 384, + "Ġ112": 385, + "Ġ46": 386, + "Ġ24": 387, + "ĠKs": 388, + "ĠQh": 389, + "ĠJc": 390, + "ĠQs": 391, + "ĠKd": 392, + "ĠQc": 393, + "ĠJd": 394, + "Ac": 395, + "ĠJs": 396, + "ĠJh": 397, + "Ġ115": 398, + "As": 399, + "Ġ97": 400, + "Ad": 401, + "Ġ113": 402, + "Ġ52": 403, + "Ġ114": 404, + "Jh": 405, + "Kd": 406, + "Ġ118": 407, + "Ġ54": 408, + "Ġ53": 409, + "Ġ116": 410, + "Kh": 411, + "Qd": 412, + "ER": 413, + "IV": 414, + "RIV": 415, + "RIVER": 416, + "Ks": 417, + "Ġ35": 418, + "Ġ119": 419, + "Qs": 420, + "Ġ55": 421, + "Qc": 422, + "Jc": 423, + "Qh": 424, + "Kc": 425, + "Jd": 426, + "Ġ25": 427, + "Ġ117": 428, + "Ġ26": 429, + "Ġ121": 430, + "Ġ27": 431, + "Js": 432, + "Ġ96": 433, + "Ġ57": 434, + "Ġ56": 435, + "Ġ34": 436, + "Ġ33": 437, + "Ġ120": 438, + "Ġ123": 439, + "Ġ31": 440, + "Ġ122": 441, + "Ġ29": 442, + "Ġ127": 443, + "Ġ28": 444, + "Ġ58": 445, + "Ġ30": 446, + "Ġ129": 447, + "Ġ60": 448, + "Ġ126": 449, + "Ġ124": 450, + "Ġ128": 451, + "Ġ59": 452, + "Ġ125": 453, + "Ġ62": 454, + "Ġ95": 455, + "Ġ32": 456, + "Ġ130": 457, + "Ġ61": 458, + "Ġ94": 459, + "Ġ134": 460, + "IN": 461, + "ALLIN": 462, + "Ġ64": 463, + "Ġ65": 464, + "Ġ137": 465, + "Ġ132": 466, + "Ġ131": 467, + "Ġ63": 468, + "Ġ133": 469, + "Ġ135": 470, + "Ġ76": 471, + "Ġ66": 472, + "Ġ73": 473, + "Ġ140": 474, + "Ġ142": 475, + "Ġ138": 476, + "Ġ75": 477, + "Ġ92": 478, + "Ġ69": 479, + "Ġ74": 480, + "Ġ67": 481, + "Ġ77": 482, + "Ġ68": 483, + "Ġ90": 484, + "Ġ93": 485, + "Ġ91": 486, + "Ġ84": 487, + "Ġ71": 488, + "Ġ141": 489, + "Ġ136": 490, + "Ġ139": 491, + "Ġ79": 492, + "Ġ78": 493, + "Ġ89": 494, + "Ġ72": 495, + "Ġ146": 496, + "Ġ85": 497, + "Ġ143": 498, + "Ġ86": 499, + "Ġ80": 500, + "Ġ144": 501, + "Ġ83": 502, + "Ġ150": 503, + "Ġ81": 504, + "Ġ147": 505, + "Ġ88": 506, + "Ġ145": 507, + "Ġ70": 508, + "Ġ87": 509, + "Ġ149": 510, + "Ġ82": 511, + "Ġ148": 512, + "Ġ151": 513, + "Ġ152": 514, + "Ġ153": 515, + "Ġ156": 516, + "12": 517, + "Ġ154": 518, + "Ġ163": 519, + "Ġ158": 520, + "Ġ160": 521, + "Ġ157": 522, + "11": 523, + "Ġ159": 524, + "Ġ165": 525, + "Ġ155": 526, + "Ġ166": 527, + "Ġ162": 528, + "Ġ176": 529, + "Ġ161": 530, + "Ġ164": 531, + "Ġ168": 532, + "Ġ172": 533, + "ĠALLIN": 534, + "Ġ173": 535, + "15": 536, + "Ġ194": 537, + "Ġ169": 538, + "Ġ186": 539, + "Ġ170": 540, + "Ġ195": 541, + "13": 542, + "Ġ175": 543, + "Ġ199": 544, + "Ġ185": 545, + "Ġ193": 546, + "Ġ187": 547, + "Ġ171": 548, + "Ġ167": 549, + "Ġ192": 550, + "Ġ177": 551, + "Ġ196": 552, + "Ġ184": 553, + "Ġ174": 554, + "Ġ197": 555, + "Ġ190": 556, + "Ġ183": 557, + "14": 558, + "Ġ178": 559, + "Ġ179": 560, + "Ġ180": 561, + "Ġ188": 562, + "Ġ191": 563, + "Ġ203": 564, + "Ġ204": 565, + "Ġ182": 566, + "Ġ198": 567, + "Ġ189": 568, + "Ġ202": 569, + "Ġ201": 570, + "Ġ206": 571, + "Ġ200": 572, + "Ġ211": 573, + "16": 574, + "Ġ207": 575, + "Ġ181": 576, + "Ġ209": 577, + "Ġ215": 578, + "Ġ214": 579, + "Ġ224": 580, + "Ġ208": 581, + "Ġ217": 582, + "Ġ216": 583, + "Ġ220": 584, + "Ġ213": 585, + "Ġ221": 586, + "Ġ228": 587, + "17": 588, + "Ġ212": 589, + "Ġ236": 590, + "Ġ219": 591, + "Ġ205": 592, + "Ġ210": 593, + "20": 594, + "Ġ234": 595, + "Ġ229": 596, + "Ġ225": 597, + "Ġ226": 598, + "Ġ232": 599, + "19": 600, + "Ġ223": 601, + "18": 602, + "Ġ218": 603, + "Ġ239": 604, + "Ġ222": 605, + "Ġ233": 606, + "Ġ247": 607, + "Ġ238": 608, + "Ġ231": 609, + "Ġ240": 610, + "Ġ227": 611, + "Ġ241": 612, + "Ġ230": 613, + "Ġ245": 614, + "Ġ235": 615, + "Ġ242": 616, + "Ġ244": 617, + "21": 618, + "Ġ246": 619, + "25": 620, + "Ġ249": 621, + "Ġ248": 622, + "22": 623, + "23": 624, + "Ġ237": 625, + "Ġ279": 626, + "Ġ265": 627, + "Ġ268": 628, + "Ġ256": 629, + "Ġ280": 630, + "24": 631, + "Ġ257": 632, + "Ġ273": 633, + "Ġ259": 634, + "Ġ262": 635, + "Ġ263": 636, + "Ġ266": 637, + "Ġ254": 638, + "Ġ243": 639, + "Ġ272": 640, + "Ġ278": 641, + "Ġ250": 642, + "Ġ253": 643, + "Ġ258": 644, + "Ġ252": 645, + "Ġ275": 646, + "27": 647, + "Ġ311": 648, + "Ġ312": 649, + "Ġ286": 650, + "Ġ276": 651, + "Ġ260": 652, + "26": 653, + "Ġ307": 654, + "Ġ281": 655, + "Ġ298": 656, + "Ġ255": 657, + "Ġ283": 658, + "28": 659, + "31": 660, + "Ġ251": 661, + "Ġ267": 662, + "Ġ282": 663, + "Ġ294": 664, + "Ġ264": 665, + "30": 666, + "Ġ269": 667, + "Ġ287": 668, + "29": 669, + "Ġ270": 670, + "Ġ285": 671, + "Ġ301": 672, + "Ġ360": 673, + "Ġ261": 674, + "Ġ317": 675, + "Ġ300": 676, + "Ġ296": 677, + "Ġ303": 678, + "Ġ271": 679, + "Ġ274": 680, + "Ġ277": 681, + "Ġ292": 682, + "Ġ305": 683, + "Ġ284": 684, + "Ġ290": 685, + "Ġ295": 686, + "Ġ319": 687, + "Ġ291": 688, + "Ġ304": 689, + "34": 690, + "37": 691, + "Ġ293": 692, + "32": 693, + "Ġ323": 694, + "36": 695, + "Ġ288": 696, + "Ġ289": 697, + "Ġ309": 698, + "Ġ310": 699, + "Ġ321": 700, + "Ġ318": 701, + "Ġ306": 702, + "Ġ329": 703, + "Ġ424": 704, + "Ġ314": 705, + "Ġ299": 706, + "Ġ361": 707, + "Ġ356": 708, + "Ġ331": 709, + "33": 710, + "Ġ351": 711, + "Ġ313": 712, + "Ġ373": 713, + "Ġ316": 714, + "Ġ327": 715, + "Ġ372": 716, + "Ġ315": 717, + "Ġ324": 718, + "Ġ411": 719, + "Ġ297": 720, + "Ġ308": 721, + "Ġ382": 722, + "Ġ302": 723, + "Ġ386": 724, + "Ġ325": 725, + "Ġ363": 726, + "Ġ357": 727, + "Ġ349": 728, + "Ġ375": 729, + "Ġ413": 730, + "Ġ359": 731, + "Ġ332": 732, + "43": 733, + "Ġ392": 734, + "Ġ334": 735, + "Ġ326": 736, + "35": 737, + "Ġ408": 738, + "Ġ464": 739, + "Ġ340": 740, + "Ġ330": 741, + "Ġ337": 742, + "Ġ320": 743, + "Ġ387": 744, + "Ġ388": 745, + "Ġ434": 746, + "Ġ341": 747, + "Ġ390": 748, + "Ġ399": 749, + "Ġ380": 750, + "Ġ367": 751, + "Ġ338": 752, + "40": 753, + "Ġ384": 754, + "Ġ432": 755, + "Ġ374": 756, + "Ġ358": 757, + "Ġ400": 758, + "Ġ401": 759, + "Ġ362": 760, + "Ġ350": 761, + "Ġ347": 762, + "Ġ333": 763, + "Ġ335": 764, + "38": 765, + "Ġ396": 766, + "Ġ404": 767, + "Ġ409": 768, + "Ġ383": 769, + "Ġ437": 770, + "Ġ364": 771, + "Ġ368": 772, + "Ġ547": 773, + "Ġ342": 774, + "41": 775, + "Ġ394": 776, + "Ġ385": 777, + "Ġ414": 778, + "Ġ425": 779, + "Ġ431": 780, + "Ġ438": 781, + "Ġ339": 782, + "Ġ406": 783, + "Ġ419": 784, + "Ġ420": 785, + "Ġ440": 786, + "Ġ352": 787, + "39": 788, + "Ġ428": 789, + "Ġ441": 790, + "Ġ353": 791, + "Ġ322": 792, + "Ġ379": 793, + "Ġ365": 794, + "45": 795, + "47": 796, + "Ġ410": 797, + "Ġ345": 798, + "Ġ348": 799, + "48": 800, + "Ġ395": 801, + "Ġ403": 802, + "Ġ426": 803, + "Ġ435": 804, + "Ġ370": 805, + "Ġ369": 806, + "Ġ468": 807, + "Ġ354": 808, + "Ġ355": 809, + "Ġ402": 810, + "Ġ377": 811, + "Ġ344": 812, + "Ġ336": 813, + "46": 814, + "Ġ398": 815, + "Ġ422": 816, + "Ġ423": 817, + "Ġ429": 818, + "Ġ436": 819, + "Ġ442": 820, + "Ġ448": 821, + "Ġ456": 822, + "Ġ484": 823, + "Ġ470": 824, + "Ġ472": 825, + "Ġ391": 826, + "Ġ393": 827, + "Ġ397": 828, + "Ġ381": 829, + "Ġ417": 830, + "Ġ430": 831, + "Ġ366": 832, + "Ġ526": 833, + "42": 834, + "44": 835, + "57": 836, + "59": 837, + "Ġ444": 838, + "Ġ449": 839, + "Ġ450": 840, + "Ġ451": 841, + "Ġ453": 842, + "Ġ486": 843, + "Ġ546": 844, + "Ġ343": 845, + "49": 846, + "58": 847, + "Ġ412": 848, + "Ġ371": 849, + "Ġ548": 850, + "Ġ328": 851, + "51": 852, + "54": 853, + "61": 854, + "63": 855, + "Ġ503": 856, + "Ġ376": 857, + "Ġ480": 858, + "Ġ466": 859, + "Ġ346": 860, + "50": 861, + "79": 862, + "Ġ407": 863, + "Ġ415": 864, + "Ġ496": 865, + "Ġ446": 866, + "Ġ455": 867, + "Ġ483": 868, + "Ġ462": 869, + "Ġ467": 870, + "Ġ553": 871, + "56": 872, + "67": 873, + "Ġ502": 874, + "Ġ416": 875, + "Ġ439": 876, + "Ġ494": 877, + "Ġ497": 878, + "Ġ498": 879, + "Ġ511": 880, + "Ġ454": 881, + "Ġ457": 882, + "Ġ458": 883, + "Ġ477": 884, + "Ġ463": 885, + "62": 886, + "68": 887, + "78": 888, + "Ġ405": 889, + "Ġ506": 890, + "Ġ378": 891, + "Ġ471": 892, + "Ġ460": 893, + "Ġ461": 894, + "Ġ577": 895 + }, + "merges": [ + [ + "Ġ", + "P" + ], + [ + "B", + "B" + ], + [ + "Ġ", + "1" + ], + [ + "Ġ", + "[" + ], + [ + "F", + "O" + ], + [ + "L", + "D" + ], + [ + "FO", + "LD" + ], + [ + "T", + "A" + ], + [ + "Ġ", + "FOLD" + ], + [ + "O", + "N" + ], + [ + "C", + "K" + ], + [ + "R", + "A" + ], + [ + "Ġ1", + "0" + ], + [ + "O", + "T" + ], + [ + "S", + "TA" + ], + [ + "ĠP", + "OT" + ], + [ + "CK", + "S" + ], + [ + "STA", + "CKS" + ], + [ + "Ġ", + "3" + ], + [ + "F", + "L" + ], + [ + "O", + "P" + ], + [ + "FL", + "OP" + ], + [ + "Ġ", + "2" + ], + [ + "Ġ", + "B" + ], + [ + "Ġ", + "4" + ], + [ + "Ġ", + "0" + ], + [ + ":", + "," + ], + [ + "B", + "L" + ], + [ + "C", + "ON" + ], + [ + "E", + "FLOP" + ], + [ + "F", + "I" + ], + [ + "G", + "U" + ], + [ + "I", + "ON" + ], + [ + "P", + "R" + ], + [ + "S", + "B" + ], + [ + "T", + "N" + ], + [ + "T", + "ION" + ], + [ + "Ġ", + "BB" + ], + [ + "Ġ", + "SB" + ], + [ + "TA", + "BL" + ], + [ + "RA", + "TION" + ], + [ + "ĠB", + "TN" + ], + [ + "CON", + "FI" + ], + [ + "GU", + "RATION" + ], + [ + "PR", + "EFLOP" + ], + [ + "TABL", + "E" + ], + [ + "CONFI", + "GURATION" + ], + [ + "Ġ", + "C" + ], + [ + "I", + "S" + ], + [ + "RA", + "IS" + ], + [ + "RAIS", + "E" + ], + [ + "Ġ", + "5" + ], + [ + "Ġ", + "9" + ], + [ + "Ġ", + "RAISE" + ], + [ + "A", + "L" + ], + [ + "AL", + "L" + ], + [ + "Ġ10", + "0" + ], + [ + "ĠC", + "ALL" + ], + [ + "E", + "CK" + ], + [ + "H", + "ECK" + ], + [ + "Ġ", + "6" + ], + [ + "Ġ1", + "1" + ], + [ + "Ġ", + "7" + ], + [ + "]", + "[" + ], + [ + "Ġ", + "8" + ], + [ + "ĠC", + "HECK" + ], + [ + "Ġ3", + "9" + ], + [ + "Ġ1", + "2" + ], + [ + "Ġ", + "A" + ], + [ + "Ġ1", + "3" + ], + [ + "E", + "T" + ], + [ + "Ġ", + "K" + ], + [ + "Ġ", + "Q" + ], + [ + "Ġ4", + "0" + ], + [ + "Ġ1", + "4" + ], + [ + "Ġ", + "J" + ], + [ + "1", + "0" + ], + [ + "Ġ10", + "1" + ], + [ + "Ġ9", + "9" + ], + [ + "ĠB", + "ET" + ], + [ + "Ġ1", + "9" + ], + [ + "Ġ1", + "5" + ], + [ + "Ġ3", + "8" + ], + [ + "Ġ5", + "0" + ], + [ + "Ġ1", + "6" + ], + [ + "R", + "N" + ], + [ + "T", + "U" + ], + [ + "TU", + "RN" + ], + [ + "C", + "HECK" + ], + [ + "Ġ1", + "8" + ], + [ + "Ġ1", + "7" + ], + [ + "Ġ10", + "3" + ], + [ + "Ġ2", + "0" + ], + [ + "Ġ10", + "2" + ], + [ + "Ġ4", + "1" + ], + [ + "Ġ10", + "4" + ], + [ + "Ġ4", + "2" + ], + [ + "Ġ4", + "3" + ], + [ + "Ġ3", + "7" + ], + [ + "Ġ4", + "9" + ], + [ + "Ġ10", + "5" + ], + [ + "C", + "ALL" + ], + [ + "Ġ2", + "1" + ], + [ + "Ġ10", + "6" + ], + [ + "Ġ4", + "4" + ], + [ + "Ġ2", + "2" + ], + [ + "Ġ9", + "8" + ], + [ + "Ġ10", + "7" + ], + [ + "ĠA", + "c" + ], + [ + "Ġ5", + "1" + ], + [ + "B", + "ET" + ], + [ + "Ġ10", + "8" + ], + [ + "ĠA", + "d" + ], + [ + "Ġ3", + "6" + ], + [ + "ĠA", + "h" + ], + [ + "Ġ2", + "3" + ], + [ + "ĠK", + "h" + ], + [ + "Ġ10", + "9" + ], + [ + "Ġ4", + "5" + ], + [ + "Ġ4", + "8" + ], + [ + "ĠA", + "s" + ], + [ + "A", + "h" + ], + [ + "Ġ11", + "0" + ], + [ + "ĠK", + "c" + ], + [ + "Ġ11", + "1" + ], + [ + "Ġ4", + "7" + ], + [ + "ĠQ", + "d" + ], + [ + "Ġ11", + "2" + ], + [ + "Ġ4", + "6" + ], + [ + "Ġ2", + "4" + ], + [ + "ĠK", + "s" + ], + [ + "ĠQ", + "h" + ], + [ + "ĠJ", + "c" + ], + [ + "ĠQ", + "s" + ], + [ + "ĠK", + "d" + ], + [ + "ĠQ", + "c" + ], + [ + "ĠJ", + "d" + ], + [ + "A", + "c" + ], + [ + "ĠJ", + "s" + ], + [ + "ĠJ", + "h" + ], + [ + "Ġ11", + "5" + ], + [ + "A", + "s" + ], + [ + "Ġ9", + "7" + ], + [ + "A", + "d" + ], + [ + "Ġ11", + "3" + ], + [ + "Ġ5", + "2" + ], + [ + "Ġ11", + "4" + ], + [ + "J", + "h" + ], + [ + "K", + "d" + ], + [ + "Ġ11", + "8" + ], + [ + "Ġ5", + "4" + ], + [ + "Ġ5", + "3" + ], + [ + "Ġ11", + "6" + ], + [ + "K", + "h" + ], + [ + "Q", + "d" + ], + [ + "E", + "R" + ], + [ + "I", + "V" + ], + [ + "R", + "IV" + ], + [ + "RIV", + "ER" + ], + [ + "K", + "s" + ], + [ + "Ġ3", + "5" + ], + [ + "Ġ11", + "9" + ], + [ + "Q", + "s" + ], + [ + "Ġ5", + "5" + ], + [ + "Q", + "c" + ], + [ + "J", + "c" + ], + [ + "Q", + "h" + ], + [ + "K", + "c" + ], + [ + "J", + "d" + ], + [ + "Ġ2", + "5" + ], + [ + "Ġ11", + "7" + ], + [ + "Ġ2", + "6" + ], + [ + "Ġ12", + "1" + ], + [ + "Ġ2", + "7" + ], + [ + "J", + "s" + ], + [ + "Ġ9", + "6" + ], + [ + "Ġ5", + "7" + ], + [ + "Ġ5", + "6" + ], + [ + "Ġ3", + "4" + ], + [ + "Ġ3", + "3" + ], + [ + "Ġ12", + "0" + ], + [ + "Ġ12", + "3" + ], + [ + "Ġ3", + "1" + ], + [ + "Ġ12", + "2" + ], + [ + "Ġ2", + "9" + ], + [ + "Ġ12", + "7" + ], + [ + "Ġ2", + "8" + ], + [ + "Ġ5", + "8" + ], + [ + "Ġ3", + "0" + ], + [ + "Ġ12", + "9" + ], + [ + "Ġ6", + "0" + ], + [ + "Ġ12", + "6" + ], + [ + "Ġ12", + "4" + ], + [ + "Ġ12", + "8" + ], + [ + "Ġ5", + "9" + ], + [ + "Ġ12", + "5" + ], + [ + "Ġ6", + "2" + ], + [ + "Ġ9", + "5" + ], + [ + "Ġ3", + "2" + ], + [ + "Ġ13", + "0" + ], + [ + "Ġ6", + "1" + ], + [ + "Ġ9", + "4" + ], + [ + "Ġ13", + "4" + ], + [ + "I", + "N" + ], + [ + "ALL", + "IN" + ], + [ + "Ġ6", + "4" + ], + [ + "Ġ6", + "5" + ], + [ + "Ġ13", + "7" + ], + [ + "Ġ13", + "2" + ], + [ + "Ġ13", + "1" + ], + [ + "Ġ6", + "3" + ], + [ + "Ġ13", + "3" + ], + [ + "Ġ13", + "5" + ], + [ + "Ġ7", + "6" + ], + [ + "Ġ6", + "6" + ], + [ + "Ġ7", + "3" + ], + [ + "Ġ14", + "0" + ], + [ + "Ġ14", + "2" + ], + [ + "Ġ13", + "8" + ], + [ + "Ġ7", + "5" + ], + [ + "Ġ9", + "2" + ], + [ + "Ġ6", + "9" + ], + [ + "Ġ7", + "4" + ], + [ + "Ġ6", + "7" + ], + [ + "Ġ7", + "7" + ], + [ + "Ġ6", + "8" + ], + [ + "Ġ9", + "0" + ], + [ + "Ġ9", + "3" + ], + [ + "Ġ9", + "1" + ], + [ + "Ġ8", + "4" + ], + [ + "Ġ7", + "1" + ], + [ + "Ġ14", + "1" + ], + [ + "Ġ13", + "6" + ], + [ + "Ġ13", + "9" + ], + [ + "Ġ7", + "9" + ], + [ + "Ġ7", + "8" + ], + [ + "Ġ8", + "9" + ], + [ + "Ġ7", + "2" + ], + [ + "Ġ14", + "6" + ], + [ + "Ġ8", + "5" + ], + [ + "Ġ14", + "3" + ], + [ + "Ġ8", + "6" + ], + [ + "Ġ8", + "0" + ], + [ + "Ġ14", + "4" + ], + [ + "Ġ8", + "3" + ], + [ + "Ġ15", + "0" + ], + [ + "Ġ8", + "1" + ], + [ + "Ġ14", + "7" + ], + [ + "Ġ8", + "8" + ], + [ + "Ġ14", + "5" + ], + [ + "Ġ7", + "0" + ], + [ + "Ġ8", + "7" + ], + [ + "Ġ14", + "9" + ], + [ + "Ġ8", + "2" + ], + [ + "Ġ14", + "8" + ], + [ + "Ġ15", + "1" + ], + [ + "Ġ15", + "2" + ], + [ + "Ġ15", + "3" + ], + [ + "Ġ15", + "6" + ], + [ + "1", + "2" + ], + [ + "Ġ15", + "4" + ], + [ + "Ġ16", + "3" + ], + [ + "Ġ15", + "8" + ], + [ + "Ġ16", + "0" + ], + [ + "Ġ15", + "7" + ], + [ + "1", + "1" + ], + [ + "Ġ15", + "9" + ], + [ + "Ġ16", + "5" + ], + [ + "Ġ15", + "5" + ], + [ + "Ġ16", + "6" + ], + [ + "Ġ16", + "2" + ], + [ + "Ġ17", + "6" + ], + [ + "Ġ16", + "1" + ], + [ + "Ġ16", + "4" + ], + [ + "Ġ16", + "8" + ], + [ + "Ġ17", + "2" + ], + [ + "Ġ", + "ALLIN" + ], + [ + "Ġ17", + "3" + ], + [ + "1", + "5" + ], + [ + "Ġ19", + "4" + ], + [ + "Ġ16", + "9" + ], + [ + "Ġ18", + "6" + ], + [ + "Ġ17", + "0" + ], + [ + "Ġ19", + "5" + ], + [ + "1", + "3" + ], + [ + "Ġ17", + "5" + ], + [ + "Ġ19", + "9" + ], + [ + "Ġ18", + "5" + ], + [ + "Ġ19", + "3" + ], + [ + "Ġ18", + "7" + ], + [ + "Ġ17", + "1" + ], + [ + "Ġ16", + "7" + ], + [ + "Ġ19", + "2" + ], + [ + "Ġ17", + "7" + ], + [ + "Ġ19", + "6" + ], + [ + "Ġ18", + "4" + ], + [ + "Ġ17", + "4" + ], + [ + "Ġ19", + "7" + ], + [ + "Ġ19", + "0" + ], + [ + "Ġ18", + "3" + ], + [ + "1", + "4" + ], + [ + "Ġ17", + "8" + ], + [ + "Ġ17", + "9" + ], + [ + "Ġ18", + "0" + ], + [ + "Ġ18", + "8" + ], + [ + "Ġ19", + "1" + ], + [ + "Ġ20", + "3" + ], + [ + "Ġ20", + "4" + ], + [ + "Ġ18", + "2" + ], + [ + "Ġ19", + "8" + ], + [ + "Ġ18", + "9" + ], + [ + "Ġ20", + "2" + ], + [ + "Ġ20", + "1" + ], + [ + "Ġ20", + "6" + ], + [ + "Ġ20", + "0" + ], + [ + "Ġ21", + "1" + ], + [ + "1", + "6" + ], + [ + "Ġ20", + "7" + ], + [ + "Ġ18", + "1" + ], + [ + "Ġ20", + "9" + ], + [ + "Ġ21", + "5" + ], + [ + "Ġ21", + "4" + ], + [ + "Ġ22", + "4" + ], + [ + "Ġ20", + "8" + ], + [ + "Ġ21", + "7" + ], + [ + "Ġ21", + "6" + ], + [ + "Ġ22", + "0" + ], + [ + "Ġ21", + "3" + ], + [ + "Ġ22", + "1" + ], + [ + "Ġ22", + "8" + ], + [ + "1", + "7" + ], + [ + "Ġ21", + "2" + ], + [ + "Ġ23", + "6" + ], + [ + "Ġ21", + "9" + ], + [ + "Ġ20", + "5" + ], + [ + "Ġ2", + "10" + ], + [ + "2", + "0" + ], + [ + "Ġ23", + "4" + ], + [ + "Ġ22", + "9" + ], + [ + "Ġ22", + "5" + ], + [ + "Ġ22", + "6" + ], + [ + "Ġ23", + "2" + ], + [ + "1", + "9" + ], + [ + "Ġ22", + "3" + ], + [ + "1", + "8" + ], + [ + "Ġ21", + "8" + ], + [ + "Ġ23", + "9" + ], + [ + "Ġ22", + "2" + ], + [ + "Ġ23", + "3" + ], + [ + "Ġ24", + "7" + ], + [ + "Ġ23", + "8" + ], + [ + "Ġ23", + "1" + ], + [ + "Ġ24", + "0" + ], + [ + "Ġ22", + "7" + ], + [ + "Ġ24", + "1" + ], + [ + "Ġ23", + "0" + ], + [ + "Ġ24", + "5" + ], + [ + "Ġ23", + "5" + ], + [ + "Ġ24", + "2" + ], + [ + "Ġ24", + "4" + ], + [ + "2", + "1" + ], + [ + "Ġ24", + "6" + ], + [ + "2", + "5" + ], + [ + "Ġ24", + "9" + ], + [ + "Ġ24", + "8" + ], + [ + "2", + "2" + ], + [ + "2", + "3" + ], + [ + "Ġ23", + "7" + ], + [ + "Ġ27", + "9" + ], + [ + "Ġ26", + "5" + ], + [ + "Ġ26", + "8" + ], + [ + "Ġ25", + "6" + ], + [ + "Ġ28", + "0" + ], + [ + "2", + "4" + ], + [ + "Ġ25", + "7" + ], + [ + "Ġ27", + "3" + ], + [ + "Ġ25", + "9" + ], + [ + "Ġ26", + "2" + ], + [ + "Ġ26", + "3" + ], + [ + "Ġ26", + "6" + ], + [ + "Ġ25", + "4" + ], + [ + "Ġ24", + "3" + ], + [ + "Ġ27", + "2" + ], + [ + "Ġ27", + "8" + ], + [ + "Ġ25", + "0" + ], + [ + "Ġ25", + "3" + ], + [ + "Ġ25", + "8" + ], + [ + "Ġ25", + "2" + ], + [ + "Ġ27", + "5" + ], + [ + "2", + "7" + ], + [ + "Ġ31", + "1" + ], + [ + "Ġ31", + "2" + ], + [ + "Ġ28", + "6" + ], + [ + "Ġ27", + "6" + ], + [ + "Ġ26", + "0" + ], + [ + "2", + "6" + ], + [ + "Ġ30", + "7" + ], + [ + "Ġ28", + "1" + ], + [ + "Ġ29", + "8" + ], + [ + "Ġ25", + "5" + ], + [ + "Ġ28", + "3" + ], + [ + "2", + "8" + ], + [ + "3", + "1" + ], + [ + "Ġ25", + "1" + ], + [ + "Ġ26", + "7" + ], + [ + "Ġ28", + "2" + ], + [ + "Ġ29", + "4" + ], + [ + "Ġ26", + "4" + ], + [ + "3", + "0" + ], + [ + "Ġ26", + "9" + ], + [ + "Ġ28", + "7" + ], + [ + "2", + "9" + ], + [ + "Ġ27", + "0" + ], + [ + "Ġ28", + "5" + ], + [ + "Ġ30", + "1" + ], + [ + "Ġ36", + "0" + ], + [ + "Ġ26", + "1" + ], + [ + "Ġ31", + "7" + ], + [ + "Ġ30", + "0" + ], + [ + "Ġ29", + "6" + ], + [ + "Ġ30", + "3" + ], + [ + "Ġ27", + "1" + ], + [ + "Ġ27", + "4" + ], + [ + "Ġ27", + "7" + ], + [ + "Ġ29", + "2" + ], + [ + "Ġ30", + "5" + ], + [ + "Ġ28", + "4" + ], + [ + "Ġ29", + "0" + ], + [ + "Ġ29", + "5" + ], + [ + "Ġ31", + "9" + ], + [ + "Ġ29", + "1" + ], + [ + "Ġ30", + "4" + ], + [ + "3", + "4" + ], + [ + "3", + "7" + ], + [ + "Ġ29", + "3" + ], + [ + "3", + "2" + ], + [ + "Ġ32", + "3" + ], + [ + "3", + "6" + ], + [ + "Ġ28", + "8" + ], + [ + "Ġ28", + "9" + ], + [ + "Ġ30", + "9" + ], + [ + "Ġ3", + "10" + ], + [ + "Ġ32", + "1" + ], + [ + "Ġ31", + "8" + ], + [ + "Ġ30", + "6" + ], + [ + "Ġ32", + "9" + ], + [ + "Ġ42", + "4" + ], + [ + "Ġ31", + "4" + ], + [ + "Ġ29", + "9" + ], + [ + "Ġ36", + "1" + ], + [ + "Ġ35", + "6" + ], + [ + "Ġ33", + "1" + ], + [ + "3", + "3" + ], + [ + "Ġ35", + "1" + ], + [ + "Ġ31", + "3" + ], + [ + "Ġ37", + "3" + ], + [ + "Ġ31", + "6" + ], + [ + "Ġ32", + "7" + ], + [ + "Ġ37", + "2" + ], + [ + "Ġ31", + "5" + ], + [ + "Ġ32", + "4" + ], + [ + "Ġ41", + "1" + ], + [ + "Ġ29", + "7" + ], + [ + "Ġ30", + "8" + ], + [ + "Ġ38", + "2" + ], + [ + "Ġ30", + "2" + ], + [ + "Ġ38", + "6" + ], + [ + "Ġ32", + "5" + ], + [ + "Ġ36", + "3" + ], + [ + "Ġ35", + "7" + ], + [ + "Ġ34", + "9" + ], + [ + "Ġ37", + "5" + ], + [ + "Ġ41", + "3" + ], + [ + "Ġ35", + "9" + ], + [ + "Ġ33", + "2" + ], + [ + "4", + "3" + ], + [ + "Ġ39", + "2" + ], + [ + "Ġ33", + "4" + ], + [ + "Ġ32", + "6" + ], + [ + "3", + "5" + ], + [ + "Ġ40", + "8" + ], + [ + "Ġ46", + "4" + ], + [ + "Ġ34", + "0" + ], + [ + "Ġ33", + "0" + ], + [ + "Ġ33", + "7" + ], + [ + "Ġ32", + "0" + ], + [ + "Ġ38", + "7" + ], + [ + "Ġ38", + "8" + ], + [ + "Ġ43", + "4" + ], + [ + "Ġ34", + "1" + ], + [ + "Ġ39", + "0" + ], + [ + "Ġ39", + "9" + ], + [ + "Ġ38", + "0" + ], + [ + "Ġ36", + "7" + ], + [ + "Ġ33", + "8" + ], + [ + "4", + "0" + ], + [ + "Ġ38", + "4" + ], + [ + "Ġ43", + "2" + ], + [ + "Ġ37", + "4" + ], + [ + "Ġ35", + "8" + ], + [ + "Ġ40", + "0" + ], + [ + "Ġ40", + "1" + ], + [ + "Ġ36", + "2" + ], + [ + "Ġ35", + "0" + ], + [ + "Ġ34", + "7" + ], + [ + "Ġ33", + "3" + ], + [ + "Ġ33", + "5" + ], + [ + "3", + "8" + ], + [ + "Ġ39", + "6" + ], + [ + "Ġ40", + "4" + ], + [ + "Ġ40", + "9" + ], + [ + "Ġ38", + "3" + ], + [ + "Ġ43", + "7" + ], + [ + "Ġ36", + "4" + ], + [ + "Ġ36", + "8" + ], + [ + "Ġ54", + "7" + ], + [ + "Ġ34", + "2" + ], + [ + "4", + "1" + ], + [ + "Ġ39", + "4" + ], + [ + "Ġ38", + "5" + ], + [ + "Ġ41", + "4" + ], + [ + "Ġ42", + "5" + ], + [ + "Ġ43", + "1" + ], + [ + "Ġ43", + "8" + ], + [ + "Ġ33", + "9" + ], + [ + "Ġ40", + "6" + ], + [ + "Ġ41", + "9" + ], + [ + "Ġ42", + "0" + ], + [ + "Ġ44", + "0" + ], + [ + "Ġ35", + "2" + ], + [ + "3", + "9" + ], + [ + "Ġ42", + "8" + ], + [ + "Ġ44", + "1" + ], + [ + "Ġ35", + "3" + ], + [ + "Ġ32", + "2" + ], + [ + "Ġ37", + "9" + ], + [ + "Ġ36", + "5" + ], + [ + "4", + "5" + ], + [ + "4", + "7" + ], + [ + "Ġ4", + "10" + ], + [ + "Ġ34", + "5" + ], + [ + "Ġ34", + "8" + ], + [ + "4", + "8" + ], + [ + "Ġ39", + "5" + ], + [ + "Ġ40", + "3" + ], + [ + "Ġ42", + "6" + ], + [ + "Ġ43", + "5" + ], + [ + "Ġ37", + "0" + ], + [ + "Ġ36", + "9" + ], + [ + "Ġ46", + "8" + ], + [ + "Ġ35", + "4" + ], + [ + "Ġ35", + "5" + ], + [ + "Ġ40", + "2" + ], + [ + "Ġ37", + "7" + ], + [ + "Ġ34", + "4" + ], + [ + "Ġ33", + "6" + ], + [ + "4", + "6" + ], + [ + "Ġ39", + "8" + ], + [ + "Ġ42", + "2" + ], + [ + "Ġ42", + "3" + ], + [ + "Ġ42", + "9" + ], + [ + "Ġ43", + "6" + ], + [ + "Ġ44", + "2" + ], + [ + "Ġ44", + "8" + ], + [ + "Ġ45", + "6" + ], + [ + "Ġ48", + "4" + ], + [ + "Ġ47", + "0" + ], + [ + "Ġ47", + "2" + ], + [ + "Ġ39", + "1" + ], + [ + "Ġ39", + "3" + ], + [ + "Ġ39", + "7" + ], + [ + "Ġ38", + "1" + ], + [ + "Ġ41", + "7" + ], + [ + "Ġ43", + "0" + ], + [ + "Ġ36", + "6" + ], + [ + "Ġ52", + "6" + ], + [ + "4", + "2" + ], + [ + "4", + "4" + ], + [ + "5", + "7" + ], + [ + "5", + "9" + ], + [ + "Ġ44", + "4" + ], + [ + "Ġ44", + "9" + ], + [ + "Ġ45", + "0" + ], + [ + "Ġ45", + "1" + ], + [ + "Ġ45", + "3" + ], + [ + "Ġ48", + "6" + ], + [ + "Ġ54", + "6" + ], + [ + "Ġ34", + "3" + ], + [ + "4", + "9" + ], + [ + "5", + "8" + ], + [ + "Ġ41", + "2" + ], + [ + "Ġ37", + "1" + ], + [ + "Ġ54", + "8" + ], + [ + "Ġ32", + "8" + ], + [ + "5", + "1" + ], + [ + "5", + "4" + ], + [ + "6", + "1" + ], + [ + "6", + "3" + ], + [ + "Ġ50", + "3" + ], + [ + "Ġ37", + "6" + ], + [ + "Ġ48", + "0" + ], + [ + "Ġ46", + "6" + ], + [ + "Ġ34", + "6" + ], + [ + "5", + "0" + ], + [ + "7", + "9" + ], + [ + "Ġ40", + "7" + ], + [ + "Ġ41", + "5" + ], + [ + "Ġ49", + "6" + ], + [ + "Ġ44", + "6" + ], + [ + "Ġ45", + "5" + ], + [ + "Ġ48", + "3" + ], + [ + "Ġ46", + "2" + ], + [ + "Ġ46", + "7" + ], + [ + "Ġ55", + "3" + ], + [ + "5", + "6" + ], + [ + "6", + "7" + ], + [ + "Ġ50", + "2" + ], + [ + "Ġ41", + "6" + ], + [ + "Ġ43", + "9" + ], + [ + "Ġ49", + "4" + ], + [ + "Ġ49", + "7" + ], + [ + "Ġ49", + "8" + ], + [ + "Ġ51", + "1" + ], + [ + "Ġ45", + "4" + ], + [ + "Ġ45", + "7" + ], + [ + "Ġ45", + "8" + ], + [ + "Ġ47", + "7" + ], + [ + "Ġ46", + "3" + ], + [ + "6", + "2" + ], + [ + "6", + "8" + ], + [ + "7", + "8" + ], + [ + "Ġ40", + "5" + ], + [ + "Ġ50", + "6" + ], + [ + "Ġ37", + "8" + ], + [ + "Ġ47", + "1" + ], + [ + "Ġ46", + "0" + ], + [ + "Ġ46", + "1" + ], + [ + "Ġ57", + "7" + ] + ] + } +} \ No newline at end of file diff --git a/tokenizer/tokenizer_config.json b/tokenizer/tokenizer_config.json new file mode 100644 index 0000000..c7330f2 --- /dev/null +++ b/tokenizer/tokenizer_config.json @@ -0,0 +1,8 @@ +{ + "backend": "tokenizers", + "bos_token": "<|startoftext|>", + "eos_token": "<|startoftext|>", + "model_max_length": 1000000000000000019884624838656, + "pad_token": "<|pad|>", + "tokenizer_class": "TokenizersBackend" +} diff --git a/venv b/venv new file mode 120000 index 0000000..b694934 --- /dev/null +++ b/venv @@ -0,0 +1 @@ +.venv \ No newline at end of file