How can I evaluate MLVU test G-AVG

[yunzhuzhang0918]

I want to test MLVU test G-AVG.
How can I evaluate MLVU test G-AVG?

[JUNJIE99]

Hi, please check the ground-truth of the test set here: https://huggingface.co/datasets/MLVU/MLVU_Test/tree/main/test-ground-truth.

The evaluation method for the generative tasks on the test set is the same as that for the dev set. You can refer to: https://github.com/JUNJIE99/MLVU/tree/main/evaluation

[yunzhuzhang0918]

Hello, Thanks for helping!
I tested our flexselect on MLVU test benchmark
and got 66.0 on MCQ benchmark, 5.93 on subscene plot, 3.83 on video summary task
Could you post my result on the leadboard?
Here is our arxiv paper:
https://arxiv.org/abs/2506.00993

Look forward to your replay

[yunzhuzhang0918]

My detail result is:
TR: 86.8 AR:59.0 NQA:70.0 ER:75.5 PQA:76.0 SQA:58.3 AO:64.3 AC:36.7 TQA:67.4 M-AVG: 66.0 SSC: 5.93 VS:3.83 G-AVG: 4.88
Model size: 72B
Input Frames: <= 1024 frames
Let me know if you need more information about this result!

[JUNJIE99]

My detail result is: TR: 86.8 AR:59.0 NQA:70.0 ER:75.5 PQA:76.0 SQA:58.3 AO:64.3 AC:36.7 TQA:67.4 M-AVG: 66.0 SSC: 5.93 VS:3.83 G-AVG: 4.88 Model size: 72B Input Frames: <= 1024 frames Let me know if you need more information about this result!

Sorry for the late reply, and thank you for submitting your model results! I have merged your PR and updated the leaderboard.

[yunzhuzhang0918]

Thanks!