Dear Authors,
Thank you for sharing your paper and code. While reviewing the variational distributions' implementations, I noticed that in PID, the standard deviation uses a tanh activation, whereas in PIB, you use softplus. Could you clarify the reason for this difference?
Was this done because the vCLUB implementation uses 2 * tanh, while VIB/DVB by [Alemi et al. 2016] uses softplus(x-5) from Eq.(19)?
Additionally, could you also please clarify why both losses are used if they seem to collapse into one at the end, if I’m not mistaken?
I would appreciate any kind of guidance for that matter.
Best,
Ilia
Dear Authors,
Thank you for sharing your paper and code. While reviewing the variational distributions' implementations, I noticed that in PID, the standard deviation uses a
tanhactivation, whereas in PIB, you usesoftplus. Could you clarify the reason for this difference?Was this done because the vCLUB implementation uses
2 * tanh, while VIB/DVB by [Alemi et al. 2016] usessoftplus(x-5)from Eq.(19)?Additionally, could you also please clarify why both losses are used if they seem to collapse into one at the end, if I’m not mistaken?
I would appreciate any kind of guidance for that matter.
Best,
Ilia