Found while reviewing #95.
Problem
Both control-gain optimization scripts evaluate the loss and auxiliary trajectory at the current parameter state, then apply an optimizer update, and finally append the updated parameters together with the pre-update loss/trajectory:
paper_results/secVd_control_gain_optimization/code/control_gain_optimization_with_collocated.pypaper_results/secVd_control_gain_optimization/code/control_gain_optimization_with_synergistic.py
This makes each saved history entry internally inconsistent. In the collocated run, the first update can produce non-finite gains while retaining the finite loss and trajectory evaluated before that update, so the later best-batch selection may save/print NaN gains as if they produced that finite rollout.
Expected fix
- Pair every loss/trajectory with the exact gain state at which it was evaluated (either store the pre-update parameters or re-evaluate after the update).
- Reject or clearly mark non-finite losses, gains, and trajectories before selecting or saving a best candidate.
- Add regression coverage that detects a one-step offset between parameter and loss history.
Found while reviewing #95.
Problem
Both control-gain optimization scripts evaluate the loss and auxiliary trajectory at the current parameter state, then apply an optimizer update, and finally append the updated parameters together with the pre-update loss/trajectory:
paper_results/secVd_control_gain_optimization/code/control_gain_optimization_with_collocated.pypaper_results/secVd_control_gain_optimization/code/control_gain_optimization_with_synergistic.pyThis makes each saved history entry internally inconsistent. In the collocated run, the first update can produce non-finite gains while retaining the finite loss and trajectory evaluated before that update, so the later best-batch selection may save/print NaN gains as if they produced that finite rollout.
Expected fix