Problems with the training and with the plot of the results

[riccardosalvucci99]

Hi, I'm using the code for the transfer nilm.
When I run the seq2point_train.py with the dataset refit for the washing machine I obtain this result:

Epoch 5 of 5 --------------------------
2023-10-16 21:50:53,666 [INFO ] loss_ep: 0.021295
2023-10-16 21:50:53,666 [INFO ] Epoch 5 of 5 took 18.028538s
2023-10-16 21:50:53,666 [INFO ] Validation...
2023-10-16 21:50:58,571 [INFO ] train loss/n_batch_train: 0.020180
2023-10-16 21:50:58,571 [INFO ] train loss: 0.201802, n_batch_train: 10
2023-10-16 21:51:03,492 [INFO ] val loss: 0.000374
2023-10-16 21:51:03,492 [INFO ] Evaluate earlystopping parameters...
2023-10-16 21:51:03,492 [INFO ] Current valid loss was 0.000374, acc was 0.000000, train loss was 0.020180, acc was 0.000000 at epoch 5.
2023-10-16 21:51:04,121 [INFO ] Best valid loss was 0.000374 and acc 0.000000 at epoch 5.
2023-10-16 21:51:04,122 [INFO ] Total training time: 141.299270s

Why is the accuracy value 0.0? Am I doing something wrong?

Then I've a problem with the plot of the results when I run the seq2point_test.py. I've no problems with the test of the network in fact I obtain the values of the parameters such as the NDE, SAE, MAE, Energy per Day but, at the end of the test, I don't see the figure with the results.

Thank you very much.

Riccardo

[MingjunZhong]

The acc was not computed because we only care about NDE, SAE, MAE, etc. For plotting the results, have you set the args.plot_results=True?

[riccardosalvucci99]

Thank you very much for the answer. Okay, I understand. So the fact that the accuracy is 0, it doesn't affect then the performance of the network right?

Yes I set arg.plot_results=True and I'm running the seq2point_test.py with Visual Studio Code

[MingjunZhong]

True - accuracy 0.00 won't affect anything.

[riccardosalvucci99]

Hello.

This morning I managed to plot the results. Nevertheless, when I run seq2point_test.py with the network trained on the washingmachine of the dataset REFIT, I obtain this result: why is the prediction line 0? These are the values of the parameters obtained with the test:
2023-10-19 11:39:10,676 [INFO ] aggregate_mean: 522
2023-10-19 11:39:10,676 [INFO ] aggregate_std: 814
2023-10-19 11:39:10,676 [INFO ] appliance_mean: 400
2023-10-19 11:39:10,676 [INFO ] appliance_std: 700
2023-10-19 11:39:10,814 [INFO ] tp=6117871.0
2023-10-19 11:39:10,814 [INFO ] fn=0.0
2023-10-19 11:39:10,814 [INFO ] 1.0
2023-10-19 11:39:10,909 [INFO ] tp=6117871.0
2023-10-19 11:39:10,909 [INFO ] fp=0.0
2023-10-19 11:39:10,909 [INFO ] 1.0
2023-10-19 11:39:10,909 [INFO ] F1:1.0
2023-10-19 11:39:10,942 [INFO ] NDE:1.0005341616229328
2023-10-19 11:39:11,505 [INFO ]
MAE: 32.52002318104955
-std: 243.8659484305266
-min: 3.051757818184342e-05
-max: 3972.002
-q1: 0.0009999999999763531
-median: 0.0009999999999763531
-q2: 5.313140869140625
2023-10-19 11:39:11,538 [INFO ] SAE: 0.88866547736957
2023-10-19 11:39:11,563 [INFO ] Energy per Day: 695.1830117782172

[MingjunZhong]

That looks like the model was not learning. Does it have the same problem for other appliances? You also need to be careful to make sure the prediction was transformed back, because when training, the data was normalised.

[riccardosalvucci99]

I've tried to test the fridge (REFIT) with the network trained on the fridge (REFIT), but I obtain the same thing. The trend of the ground truth of the fridge obviously is different from that one of the washingmachine (same thing for the aggregate).
I don't understand which is the problem, the code is exactly like yours, I've imported it.