Abnormal G1 steering behavior with SMP motion prior trained on larger retargeted LaFAN1 dataset

[yanzhuohao123]

Hi, I am trying to migrate SMP to the Unitree G1 robot, but I encountered an issue when training the steering task.

When I train the motion prior using a larger set of retargeted LaFAN1 locomotion motions:

motions:
  - file: "data/motions/lafan1_g1/run1_subject2.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/run1_subject5.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/run2_subject1.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/run2_subject4.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/sprint1_subject2.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/sprint1_subject2.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk1_subject1.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk1_subject2.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk1_subject5.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk2_subject1.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk2_subject3.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk2_subject4.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk3_subject1.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk3_subject2.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk3_subject3.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk3_subject4.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk3_subject5.pkl"
    weight: 1.0
  - file: "data/motions/lafan1_g1/walk4_subject1.pkl"
    weight: 1.0

the downstream G1 steering task produces very strange walking behavior. This happens even when I sample g1_run.pkl and g1_walk.pkl as the initial motions for the downstream task.

2026-05-29.15-36-18.mov

However, if I train the motion prior using only the following two G1-specific motions:

motions:
  - file: "data/motions/g1/g1_walk.pkl"
    weight: 1.0
  - file: "data/motions/g1/g1_run.pkl"
    weight: 1.0

the downstream steering behavior looks much more normal.

2026-06-01.09-58-35.mov

I would like to ask whether this is expected when the motion prior is trained on a larger and more diverse motion dataset. Could the larger motion prior dataset make the learned prior distribution too broad, noisy, or difficult for the downstream steering policy to use effectively, especially for a robot like Unitree G1?

I also wonder whether SMP may suffer from a similar mode collapse issue as AMP. In AMP, the policy can sometimes exploit the discriminator reward or converge to only a small subset of the demonstrated motion distribution. Does SMP have a similar failure mode when the motion prior dataset contains multiple locomotion styles, such as walking, running, and sprinting? For example, could the downstream policy collapse to an unnatural or mixed gait instead of selecting a stable walking/running mode?

[Murrol]

Thanks for trying SMP with G1.

The short answer is yes. SMP, like other distribution-matching paradigms, can be prone to mode collapse. With larger datasets, there may be many modes that satisfy the task objective. The mode learned by the policy may not be the most natural-looking one, even if that mode exists in the dataset.

Potential solutions include using a conditional version of SMP, where the desired mode can be explicitly specified, or curating the dataset to exclude undesired behaviors.

That said, this bending-over behavior may also be improved by tuning the diffusion model’s normalizer std or the per-dimension feature weights in the SDS loss, so that SMP focuses more on reducing these suboptimal behaviors.