Question

[TDL77]

Please tell me, did I get it right here?
..for some reason, the results even got worse.

import torch.nn as nn
import torch.optim as optim

class MultipleRegression(nn.Module):
def init(self, num_features):
super(MultipleRegression, self).init()

    self.layer_1 = nn.Linear(num_features, 16)
    self.layer_2 = nn.Linear(16, 32)
    self.layer_3 = nn.Linear(32, num_features)
    self.layer_out = nn.Linear(num_features, 1)
    
    self.relu = nn.ELU()  
    self.revin_layer = RevIN(num_features)   # <<<-----

def forward(self, inputs):
    x_in = self.revin_layer(inputs, 'norm')
    
    x = self.relu(self.layer_1(x_in))
    x = self.relu(self.layer_2(x))
    x = self.relu(self.layer_3(x))
    
    x = self.revin_layer(x, 'denorm')
    x_out = self.layer_out(x)

    return x_out

[ts-kim]

Hi,

RevIN is a flexible, end-to-end trainable layer that can be applied to any arbitrarily chosen layers, effectively suppressing non-stationary information (mean and variance of the instance) in one layer and restoring it in another layer at a virtually symmetric position, e.g., input and output layers.

In your case, I think RevIN is not applied to the "symmetric position", because the input inputs is normalized and the hidden space vector x is denormalized.

Also, you should check whether the distribution shift problem occurs in your dataset before adopting RevIN.

[TDL77]

Hi,

RevIN is a flexible, end-to-end trainable layer that can be applied to any arbitrarily chosen layers, effectively suppressing non-stationary information (mean and variance of the instance) in one layer and restoring it in another layer at a virtually symmetric position, e.g., input and output layers.

In your case, I think RevIN is not applied to the "symmetric position", because the input inputs is normalized and the hidden space vector x is denormalized.

Also, you should check whether the distribution shift problem occurs in your dataset before adopting RevIN.

Thank you very much for your reply.can ask you for help.

• how would you apply RevIN , in this implementation (code)
• tell me how you can determine the shift in the distribution between the layers of the network?
  Thanks in advance for your time and help.