WorldScore leaderboard scores

[terrancefox]

Thank you very much for your excellent open-source work. We are currently conducting academic research related to your project.
We have a question regarding the WorldScore results. We noticed that the scores of FantasyWorld (both FantasyWorld 1.0 and FantasyWorld 0.1) on the WorldScore leaderboard are significantly higher than those reported in your paper, especially on the Object Control and Content Alignment metrics. Similarly, the scores for Voyager reported in your paper are considerably lower than the Voyager scores shown on the WorldScore leaderboard.

We also ran the WorldScore static sequences using your open-source FantasyWorld-Wan2.1-I2V-14B-480P model, and the scores we obtained are consistent with the results in your paper.
We would appreciate your clarification on whether:

• The scores on the leaderboard were computed using a newer model, or
• Some bug in the WorldScore evaluation script was fixed.

[jaimeyzzz]

Thank you very much for your excellent open-source work. We are currently conducting academic research related to your project. We have a question regarding the WorldScore results. We noticed that the scores of FantasyWorld (both FantasyWorld 1.0 and FantasyWorld 0.1) on the WorldScore leaderboard are significantly higher than those reported in your paper, especially on the Object Control and Content Alignment metrics. Similarly, the scores for Voyager reported in your paper are considerably lower than the Voyager scores shown on the WorldScore leaderboard.

We also ran the WorldScore static sequences using your open-source FantasyWorld-Wan2.1-I2V-14B-480P model, and the scores we obtained are consistent with the results in your paper. We would appreciate your clarification on whether:

• The scores on the leaderboard were computed using a newer model, or
• Some bug in the WorldScore evaluation script was fixed.

Hi @terrancefox, I can get matching quality metrics, but my Camera & Object Control scores are lower than the paper. Did you use any special configs/preprocessing for control evaluation when reproducing? Thanks a lot!