Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ See [RESEARCH.md](RESEARCH.md) for research details, [plans/002-ml-model.md](pla

- **Architecture:** Encoder-decoder transformer (44.5M default, 6.5M medium, 1M small)
- **Audio input:** Log-mel spectrogram (80 bins, sr=22050, hop=512), beat-aligned to 1/16th note grid. Optional onset strength channel (+1 bin).
- **Positional encoding:** Sinusoidal (default) or RoPE (config.use_rope). RoPE applied to self-attention Q/K only (not cross-attention).
- **Positional encoding:** RoPE (default) or sinusoidal (config.use_rope=False). RoPE applied to self-attention Q/K only (not cross-attention).
- **Scope:** Color notes only (no bombs, walls, arcs, chains until core model performs well)
- **Output:** Beat-quantized compound tokens (291 vocab) → v2 Beat Saber JSON
- **Token format:** `START DIFF BAR POS LEFT RIGHT ... BAR ... END`
Expand All @@ -92,7 +92,7 @@ Best model after 16 epochs: **val_loss=2.055, 60.6% token accuracy**. Model plat

## Open Questions

- **Medium model training:** Ready to train with `configs/medium.json` — 6.5M params, Expert+ only, onset features, fits 8GB VRAM
- **RoPE evaluation:** RoPE implemented but disabled by default (use_rope=False). Enable and compare against sinusoidal PE.
- **Medium model training:** Ready to train with `configs/medium.json` — 6.5M params, Expert+ only, onset features, RoPE enabled, fits 8GB VRAM
- **RoPE evaluation:** RoPE now default for all configs. Compare against sinusoidal PE baseline after training.
- **Feedback capture system:** In-game mechanism to collect player feedback (later phase)
- **RL fine-tuning:** After supervised pretraining, fine-tune with player feedback reward model
2 changes: 1 addition & 1 deletion beat_weaver/model/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ class ModelConfig:
use_onset_features: bool = False # Concatenate onset strength as extra mel channel

# Positional encoding
use_rope: bool = False # Use RoPE instead of sinusoidal PE
use_rope: bool = True # Use RoPE instead of sinusoidal PE

# Auxiliary losses
density_loss_weight: float = 0.1
Expand Down
2 changes: 1 addition & 1 deletion configs/medium.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"density_loss_weight": 0.1,
"color_balance_weight": 0.1,
"use_onset_features": true,
"use_rope": false,
"use_rope": true,
"min_difficulty": "Expert",
"characteristics": ["Standard"],
"min_bpm": 50.0,
Expand Down
3 changes: 2 additions & 1 deletion configs/small.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,6 @@
"gradient_accumulation_steps": 1,
"early_stopping_patience": 10,
"official_ratio": 0.2,
"density_loss_weight": 0.1
"density_loss_weight": 0.1,
"use_rope": true
}