Hello, thank you for sharing your work. I have some questions regarding the reproduction of results, specifically for the pruned model fine-tuning.
I followed your provided train_masked_kd.py script to reproduce the results reported in Table 1 for TinyDiT-D14 (KD) 100K. Since my machine supports a larger batch size than the default script, I measured training progress in terms of epochs. I trained the pruned model for 25 epochs, which I believe should be roughly equivalent to 100k iterations.
However, the performance I obtained was far worse than expected. For evaluation, I used both cfg=1.5 (in sample_ddp.py) and cfg=4.0 (in sample.py) settings. In contrast to the paper’s reported results (IS = 207.27, FID = 3.73), my results were:
- cfg=4.0 → IS = 121.14, FID = 21.63
- cfg=1.5 → IS = 16.8, FID = 100.95
For training and evaluation, I strictly followed the scripts you provided, with the only modification being a larger batch size. When reproducing the pruning procedure using prune_by_learning.py, my results were highly consistent with those reported in the paper. However, the fine-tuning stage after pruning performed very poorly in my experiments.
I am wondering whether there might be something wrong in my setup or training procedure. Could you provide guidance on what might be causing this discrepancy?
Thank you very much for your time and support!
Hello, thank you for sharing your work. I have some questions regarding the reproduction of results, specifically for the pruned model fine-tuning.
I followed your provided
train_masked_kd.pyscript to reproduce the results reported in Table 1 for TinyDiT-D14 (KD) 100K. Since my machine supports a larger batch size than the default script, I measured training progress in terms of epochs. I trained the pruned model for 25 epochs, which I believe should be roughly equivalent to 100k iterations.However, the performance I obtained was far worse than expected. For evaluation, I used both
cfg=1.5(insample_ddp.py) andcfg=4.0(insample.py) settings. In contrast to the paper’s reported results (IS = 207.27, FID = 3.73), my results were:For training and evaluation, I strictly followed the scripts you provided, with the only modification being a larger batch size. When reproducing the pruning procedure using
prune_by_learning.py, my results were highly consistent with those reported in the paper. However, the fine-tuning stage after pruning performed very poorly in my experiments.I am wondering whether there might be something wrong in my setup or training procedure. Could you provide guidance on what might be causing this discrepancy?
Thank you very much for your time and support!