Why can the two scale models be used directly together without training?

[IzumiKDl]

Thank you so much for your amazing work! When running the code, we noticed that two VAR models of different scales are directly employed as the drafter and refiner models without additional training. This raises the question: why can these two models be used together seamlessly without further fine-tuning?

[czg1225]

Hi @IzumiKDl ,
Thanks for your interest! Although the drafter and refiner are two different VAR models, they use the same VQVAE encoder during the training phase. This means that they have a shared discrete latent space. As a result, they can conduct collaborative decoding without training. It is worth mentioning that fine-tuning the model on their specialized scales will further improve the generation effect.