Difference between act(y) and act(yref) causes errors when the activation function is LeakyRelu

[iotg-ba-hw-acceleration]

Hi,
Inside the class SigmaDeltaNeuronsInstErrExec, the use of "self.process.act_fn(y) - self.process.act_fn(vars.y_ref)" to compute delta_a causes errors when the activation function is LeakyReLU. This happens because, delta_a is used to update y_ref. Consider a time t when the value of y becomes negative and the slope of leakyrelu for negative values be 0.1, the value of delta_a correctly corresponds to a value (y(t)0.1 - y_ref(t)) and it update y_ref(t) to reflect the value of 0.1y. But in the next time step when y_ref is passed to the activation function, the output becomes 0.01*y(t) and the absolute value of delta_a is more. Instead of applying the activation function over y_ref, a direct subtraction is sufficient and it works for ReLU and other non-linear activation functions also.
I have added the section of code which worked for me.

delta_a = tf.cond(
tf.logical_and(
tf.greater_equal(self.time, self.process.first_valid_ts),
tf.equal(tf.math.mod(self.time
- self.process.first_valid_ts + 1,
self.process.out_decim_interval), 0)),
true_fn=lambda: self.process.act_fn(y)
- vars.y_ref,
false_fn=lambda: tf.zeros_like(vars.delta_a))