[M7] Hyperparameter Sensitivity Analysis (Table 5)

[timwei0801]

Background

Reviewers expect analysis of how key hyperparameters affect model performance. Add a new subsection in Section 5.2 with Table 5.

Requirements

Hyperparameters to Sweep

Parameter	Values to Test	Default
Hidden dimension (D)	32, 64, 128	64
Attention heads (K)	4, 8, 16	8
Learning rate	0.0005, 0.001, 0.002	0.001

• Bold = default value (already trained)
• Total: 7 unique configurations (3 + 3 + 3, minus 2 overlapping defaults)

Metrics per Configuration

• MAE, RMSE, R²

Paper Location

• Section: 5.2 (new subsection)
• Artifact: Table 5 — Hyperparameter Sensitivity Analysis

Technical Details

1. Create experiments/hyperparam_sweep.py:

   • Define configurations as list of dicts
   • Train each config (100 epochs, early stopping)
   • Record test metrics
   • Save to experiments/hyperparam_results.json

2. Note: When changing D, K must satisfy D % K == 0

   • D=32, K=8 → d=4 ✓
   • D=128, K=8 → d=16 ✓
   • D=64, K=4 → d=16 ✓
   • D=64, K=16 → d=4 ✓

3. Default run already exists — reuse results from the full training (D=64, K=8, lr=0.001)

Acceptance Criteria

• 7 configurations defined and documented
• All configurations trained to convergence
• Table 5 with MAE/RMSE/R² for each configuration
• Analysis text: which parameters are most sensitive
• Results saved to experiments/hyperparam_results.json

Estimated Effort

• Development: ~2 hours (script + table generation)
• Training: ~6-7 hours (7 configs × ~1 hour, minus 1 already done)
• Total: ~8-9 hours

Notes

Training can be automated overnight. Script should support resuming if interrupted.