Temporal Conv

[watsy1314]

Hello Author. There are some technical details I would like to ask you about the Temporal Conv operation in Figure 2 of the original article. Inputting [4, 16, 3, 224, 384] video frames to the network, we will get [4, 768/384/192/96, 9, 7/14/28/56, 12/24/48/96] feature maps after passing through the Multi-modal Attention Block. a kernel_size=5, stride=5 operation to carry out, but my experiments afterward found that if 8, 9 or some other combinations of kernel_size and stride are used to let the time step 9 reduce to 1, all of them will have some metrics abnormally high in the training set, while the validation set these metrics and the AUC are exceptionally low, may I ask what this is Why? And kernel_size=5, stride=5 doesn't cover all the time steps, the rest of the information can only be discarded? Here is the corresponding code for Temporal Conv from the original article：
class ReduceTemp(nn.Module):
def init(
self,
in_chans=3,
embed_dim=64,
temporal_dim=1,
stride=1,
padding=0,
):
super().init()
self.proj = nn.Sequential(
nn.Conv3d(in_chans,
embed_dim,
kernel_size=(temporal_dim, 1, 1),
stride=(stride, 1, 1),
padding=(padding, 0, 0),
bias=False),
nn.ReLU(inplace=True),
)

def forward(self, x):
    x = self.proj(x)
    return x