The counts of correct predictions for both masked and unmasked tokens are considerably low.

[soloistzy]

Dear author,

I hope this message finds you well. I've utilized your avhubert model for pre-training tasks on a new dataset. Below is the training script I've used:

CUDA_VISIBLE_DEVICES=1 fairseq-hydra-train --config-dir /media/av_hubert/av_hubert/avhubert/conf/pretrain --config-name base_lrs3_iter1.yaml task.data=/media/aa/a4b46d17-0f49-4392-98d6-49a5c9dee8e9/data/CN-CVS/tsv task.label_dir=/media/aa/a4b46d17-0f49-4392-98d6-49a5c9dee8e9/data/CN-CVS/lab/lab1 model.label_rate=25 hydra.run.dir=/media/aa/a4b46d17-0f49-4392-98d6-49a5c9dee8e9/zzs/model/pretrain common.user_dir=`pwd`

However, upon inspecting the training logs during execution, I noticed the following output:

{'loss_m_0': 3050.0, 'loss_u_0': 3306.0, 'loss_features_pen': 3.2800943851470947, 'correct_m_0': 4, 'count_m_0': 448, 'correct_u_0': 0, 'count_u_0': 451}
{'loss': 3106.0, 'ntokens': 504, 'nsentences': 7, 'sample_size': 504, 'loss_m_0': 3102.0}

It seems that the counts of correct predictions for both masked and unmasked tokens are considerably low. Is this a normal occurrence?

Thank you for your assistance.