Where is the code corresponding to the encoder part of ISRD in the paper?

[ncepuzxy]

Could you please tell me where the code is for converting the features of CNN and ViT into the same format? Specifically, where is the code corresponding to the encoder part of ISRD in the paper?

[gyguo]

Initialization in tools_1_ts/train_isrd_5runs.py: Line 41-49

Model file: ib/models/hrir.py

[ncepuzxy]

How does Generalized Spatial Feature Preprocessing (GSFP) convert the spatial features of CNNs and ViTs into the same form? Where is the code for the dimensional transformation of ViT features?

[gyguo]

For CNN, return the original feature
For ViT, in lib/models/vit_pixel.py, Line 264-267 and Line 273-278

if return_feat:
            feats = []
            patch_tokens = x.permute(0, 2, 1).view(x.size(0), x.size(2), int(math.sqrt(x.size(1))), int(math.sqrt(x.size(1))))
            feats.append(patch_tokens)

if return_feat:
            for d in range(self.depth):
                x = self.blocks[d](x)
                patch_tokens = x[:, 1:, :].permute(0, 2, 1).view(x.size(0), x.size(2), int(math.sqrt(x.size(1))),
                                                       int(math.sqrt(x.size(1))))
                feats.append(patch_tokens)