What is the point of EMA in training scripts

[Greemwood]

Thanks for great work! I have some questions about the EMA used in the training code, as ema is not mentioned commonly in paper, what is the role of it? And it seems not used in sample code. Why print decisions and confidence using ema gumbel_gates and options instead of original model?

[Greemwood]

By the way, in the prune_by_learning.py, why reload original checkpoint for final decisions model save instead of saving the trained one?