Questions about the tutorial

[imerelli]

Hi,
I'm trying to run the tutorial, but I have this problem. In the end the script fails.
Can you help me in understanding which is the problem?

(titan) [imerelli@gpu005 tutorial]$ python3 ../scripts/flexible_training.py \
> data/train_small.csv \
> data/test_small.csv  \
> data/tcr_full.csv \
> data/epitopes.smi \
> trained_model \
> data/params_small.json \
> tutorial_setting \
> bimodal_mca

INFO:tutorial_setting:Start data preprocessing...
INFO:tutorial_setting:Ligand file has extension .smi 
Please make sure ligand is provided as SMILES.
2
ERROR:pytoda.datasets.smiles_dataset:Since you provided a smiles_language, the following parameters to this class will be ignored: canonical, augment, kekulize, all_bonds_explicit, selfies, sanitize, all_hs_explicit, remove_bonddir, remove_chirality, randomize, add_start_and_stop, padding, padding_length.
Here are the problems:
ERROR:pytoda.datasets.smiles_dataset:Provided arg add_start_and_stop:True does not match the smiles_language value: False NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:Provided arg padding:True does not match the smiles_language value: False NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:Provided arg padding_length:422 does not match the smiles_language value: None NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:To get rid of this, adapt the smiles_language *offline*, feed itready for intended usage, and adapt the constructor args to be identical with their equivalents in the language object
ERROR:pytoda.datasets.smiles_dataset:Since you provided a smiles_language, the following parameters to this class will be ignored: canonical, augment, kekulize, all_bonds_explicit, selfies, sanitize, all_hs_explicit, remove_bonddir, remove_chirality, randomize, add_start_and_stop, padding, padding_length.
Here are the problems:
ERROR:pytoda.datasets.smiles_dataset:Provided arg add_start_and_stop:True does not match the smiles_language value: False NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:Provided arg padding:True does not match the smiles_language value: False NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:Provided arg padding_length:422 does not match the smiles_language value: None NOTE: smiles_language value takes preference!!
ERROR:pytoda.datasets.smiles_dataset:To get rid of this, adapt the smiles_language *offline*, feed itready for intended usage, and adapt the constructor args to be identical with their equivalents in the language object

......

INFO:tutorial_setting:Training about to start...

INFO:tutorial_setting:== Epoch [0/100] ==
INFO:tutorial_setting:Batch 0/3
INFO:tutorial_setting:   **** TRAINING ****   Epoch [1/100], loss: 0.68310. This took 1.6 secs.
Traceback (most recent call last):
  File "/opt/tools/deg/TITAN/tutorial/../scripts/flexible_training.py", line 483, in <module>
    main(
  File "/opt/tools/deg/TITAN/tutorial/../scripts/flexible_training.py", line 399, in main
    predictions = torch.cat(predictions, dim=0).flatten().cpu().numpy()
RuntimeError: torch.cat(): expected a non-empty list of Tensors

[jannisborn]

Hi, the error message says it all, you're trying to stack an empty list of tensors. This means that your test dataset is empty or at least the test dataloader could not retrieve a batch. I dont think it's an issue with the code, more with the data you provide

[imerelli]

Uhm, ok, but this is the dataset you provided as test, maybe I'm missing something?

[jannisborn]

Your error log says test_small.csv. I may be wrong since this is an old project, but I dont remember that we distributed a test_small.csv file. If you processed/truncated this file somehow, that is your responsibility and indeed the test set might become empty, dependent also on the splitting strategy

[imerelli]

Hi, test_small.csv is in tutorial/data and this is the content.
Can you help me in fixing the error?

,Unnamed: 0,ligand_name,sequence_id,label
10,10,28,108,1
47,47,121,528,1
48,48,121,533,1
51,51,22,596,1
58,58,22,686,1
59,59,22,692,1
66,66,76,742,1
67,67,22,743,1
68,68,22,757,1
70,70,76,769,1
71,71,22,786,1
78,78,22,827,1
79,79,22,831,1
82,82,22,873,1
90,90,22,910,1
91,91,22,917,1
95,95,22,957,1
96,96,22,963,1
107,107,22,1023,1
112,112,22,1061,1
113,113,22,674,1
114,114,22,1072,1
120,120,22,1128,1
123,123,22,1151,1
124,124,22,595,1
125,125,22,1154,1
127,127,22,1161,1
132,132,22,1203,1
134,134,22,1211,1
136,136,22,1215,1
137,137,22,1223,1
152,152,22,1386,1
165,165,22,1474,1
168,168,22,1497,1
178,178,22,1578,1
182,182,76,1606,1
200,200,22,1769,1
201,201,76,1770,1
202,202,22,1780,1
205,205,22,1815,1
226,226,76,2006,1
231,231,76,2046,1
244,244,22,1934,1
257,257,76,2235,1
263,263,22,2272,1
267,267,22,2288,1
278,278,22,2411,1
287,287,22,2515,1
290,290,22,2528,1
292,292,22,2533,1
296,296,22,2567,1
297,297,22,2578,1
300,300,22,2648,1
301,301,22,2655,1
306,306,22,2563,1
307,307,22,2683,1
311,311,28,2757,1
312,312,28,2765,1
315,315,28,2775,1
317,317,28,2843,1
318,318,28,2849,1
319,319,28,2850,1
331,331,28,3023,1
332,332,28,3024,1
333,333,28,3026,1
334,334,28,3027,1
335,335,28,3039,1
336,336,28,3058,1
337,337,28,3062,1
342,342,76,3124,1
520,520,121,4666,1
521,521,121,4675,1
531,531,121,4750,1
532,532,121,4803,1
542,542,121,4913,1
546,546,121,4964,1
547,547,121,4970,1
558,558,121,5051,1
559,559,121,5056,1
560,560,121,5057,1
665,665,121,6194,1
666,666,121,6195,1
667,667,121,6203,1
668,668,121,6214,1
669,669,121,6220,1
670,670,121,6224,1
671,671,121,6250,1
672,672,121,6260,1
673,673,121,6264,1
674,674,121,6283,1
675,675,121,6292,1
676,676,121,6293,1
677,677,121,6295,1
678,678,121,6307,1
679,679,121,6327,1
680,680,121,6355,1
681,681,121,6357,1
682,682,121,6373,1
781,781,22,7279,1
785,785,76,7347,1
786,786,76,7352,1
787,787,76,7375,1
788,788,76,7386,1
789,789,76,7416,1
790,790,76,7432,1
791,791,76,7486,1
792,792,76,7487,1
793,793,76,7496,1
794,794,76,7501,1
795,795,76,7504,1
796,796,76,7505,1
797,797,76,7507,1
798,798,76,7516,1
799,799,76,7521,1
800,800,76,7546,1
801,801,76,7555,1
802,802,76,7562,1
803,803,76,7569,1
900,900,28,8610,1
901,901,28,8623,1
951,951,121,9061,1
952,952,121,9067,1
953,953,121,9069,1
954,954,121,9078,1
955,955,121,9079,1
956,956,121,9090,1
957,957,121,9104,1
958,958,121,9122,1
983,983,28,9398,1
984,984,28,9415,1
985,985,28,9418,1
986,986,28,9443,1
987,987,28,9453,1
988,988,28,9454,1
989,989,28,9462,1
990,990,28,9474,1
991,991,28,9480,1
992,992,28,9485,1
993,993,28,9486,1
1004,1004,76,9621,1
1062,1062,28,40,0
1081,1081,22,190,0
1128,1128,28,757,0
1169,1169,76,1045,0
1188,1188,121,1179,0
1208,1208,121,1348,0
1216,1216,22,1407,0
1217,1217,76,1421,0
1224,1224,121,1466,0
1234,1234,121,1524,0
1274,1274,28,1912,0
1296,1296,76,2096,0
1311,1311,22,2188,0
1316,1316,22,2221,0
1343,1343,28,2453,0
1374,1374,76,2757,0
1387,1387,121,2960,0
1393,1393,76,3026,0
1411,1411,76,3233,0
1421,1421,28,3320,0
1530,1530,28,4304,0
1560,1560,121,4539,0
1585,1585,76,4733,0
1589,1589,28,132,0
1685,1685,22,5804,0
1751,1751,121,6480,0
1759,1759,22,6584,0
1778,1778,22,6773,0
1787,1787,76,6835,0
1881,1881,22,7740,0
1896,1896,28,7866,0
1909,1909,76,8082,0
1910,1910,76,8084,0
1912,1912,76,8115,0
1923,1923,76,8237,0
1978,1978,28,8787,0
2000,2000,28,8973,0
2106,2106,22,10021,0
2118,2118,121,10158,0
```

[jannisborn]

Hi @imerelli

Basically the test dataset is smaller than the batch size and therefore
To solve this programmatically set drop_last=False in the test data loaders.

[jannisborn]

Otherwise adjust the config to set batch size to be lower than the test dataset size but beware that the last samples will be dropped in that case.
Feel free to open a PR, I would happily merge

[jannisborn]

@imerelli see #22

[imerelli]

Ok, thank you.
Do I need to clone the repository again, or is it just something in the README about how to use the code?
I couldn’t spot the changes at first glance.

[jannisborn]

You need to pull the changes from the branch. Or from main after the PR is merged, probably I do that tomorrow