Hi, I am applying SharedObservation training on Pi0.5. However, I see that the state are processing differently compared to original pi0.5.
In original Pi0.5, state is discretized and treated as text tokens and passed to VLM as a part of prefix_embs. However, when training with Shared observation config, I see that state is not processed in VLM and it is processed like a suffix embed.
Can anyone help me to clarify it? Is my understand correct?
If yes, How does this difference effect the model performance?
Hi, I am applying SharedObservation training on Pi0.5. However, I see that the state are processing differently compared to original pi0.5.
In original Pi0.5, state is discretized and treated as text tokens and passed to VLM as a part of
prefix_embs. However, when training with Shared observation config, I see that state is not processed in VLM and it is processed like a suffix embed.Can anyone help me to clarify it? Is my understand correct?
If yes, How does this difference effect the model performance?