Question regarding SFT checkpoint initialization for RL/Distillation and performance degradation in 1.7B reproduction #18

Hi authors,

Thanks for sharing this excellent work and the detailed paper!

I am currently trying to reproduce the results for the Qwen3-1.7B model following the implementation details provided in Appendix B.4. During this process, I noticed some unexpected phenomena regarding the Vanilla and SFT evaluations, and I would love to get your insights.

My Observations:
In my local reproduction on the 1.7B scale:

My Vanilla model evaluation scores are higher than those reported in Table 1.

My SFT model evaluation scores are lower than those reported in Table 1.

Most notably, the SFT model performs worse than the Vanilla model.

This performance degradation after SFT is very similar to what you reported for the 0.6B model in Table 1, but I am surprisingly observing it on the 1.7B model as well.

My Questions:

Could you confirm if the results in Table 1, as well as all subsequent RL and distillation training (OPD, SOD, GRPO), are initialized from the checkpoint after the 5 epochs of SFT mentioned in Appendix B.4?

Given the phenomenon where SFT significantly degrades the base model's performance, is it methodologically optimal/reasonable to use these degraded SFT weights as the starting point for OPD, SOD, and GRPO?

Any clarification on your experimental setup and the rationale behind initializing from this specific SFT checkpoint would be greatly appreciated. Thanks again for your time and the great contribution!