Skip to content

Fix H0.5 benchmark inference compatibility - #50

Open
MarkfuGod wants to merge 1 commit into
BeingBeyond:mainfrom
MarkfuGod:fix/h05-benchmark-inference-compat
Open

Fix H0.5 benchmark inference compatibility#50
MarkfuGod wants to merge 1 commit into
BeingBeyond:mainfrom
MarkfuGod:fix/h05-benchmark-inference-compat

Conversation

@MarkfuGod

@MarkfuGod MarkfuGod commented May 23, 2026

Copy link
Copy Markdown

Summary

  • Fix the get_modality_config inference endpoint so it is invoked without a request payload.
  • Add a generic BaseDataConfig.modality_config() implementation for client-side modality discovery.
  • Cast LIBERO and RoboCasa benchmark state observations to float32 before sending them to the H0.5 inference server.

Why

While running the released Being-H0.5 LIBERO inference server and benchmark client end-to-end, two server-side compatibility issues appeared:

  • get_modality_config was registered as a payload-taking endpoint, causing an extra argument to be passed into BeingHPolicy.get_modality_config().
  • BeingHPolicy.get_modality_config() expected data configs to expose modality_config(), but BaseDataConfig did not provide it.

The LIBERO benchmark then reached a real rollout but failed during state preprocessing because simulator observations were sent as float64, while the H0.5 transform pipeline expects consistent float32 state tensors. RoboCasa uses a similar observation path, so this patch applies the same dtype normalization there.

Test plan

  • python -m py_compile Being-H05/BeingH/inference/beingh_service.py Being-H05/configs/data_config.py Being-H05/BeingH/benchmark/utils/policy.py Being-H05/BeingH/benchmark/robocasa/run_robocasa_eval_fast.py
  • Loaded Being-H05-2B_libero with BeingHPolicy.
  • Started BeingH.inference.run_server_vla locally.
  • Verified client ping.
  • Verified client get_modality_config.
  • Verified remote dummy get_action.
  • Ran minimal LIBERO rollout:
    • suite: libero_spatial
    • task id: 0
    • trials: 1
    • result: 1/1 success
  • Ran full LIBERO evaluation with robosuite==1.4.0:
    • libero_spatial: 493/500, 98.6%
    • libero_object: 485/500, 97.0%
    • libero_goal: 490/500, 98.0%
    • libero_10: 475/500, 95.0%

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant