Not normalizing the expert states when computing gradient penalty.

[EGalahad]

https://github.com/Alescontrela/AMP_for_hardware/blob/bfb0dbdcf32bdf83a916790bddf193fffc7e79b8/rsl_rl/rsl_rl/algorithms/amp_ppo.py#L235

When using state normalization, the sample_amp_expert tuple is not normalized.

[EGalahad]

https://github.com/Alescontrela/AMP_for_hardware/blob/bfb0dbdcf32bdf83a916790bddf193fffc7e79b8/rsl_rl/rsl_rl/algorithms/amp_ppo.py#L254

and I think maybe we should update the normalizer stats before normalizing them?

[apirrone]

@EGalahad What effect do you think this has ?

[EGalahad]

I just think it is weird to pass in the normalized values to update the normalizer. I think you should pass un-normalized version so that the normalizer can update running mean?