Skip to content

eval(sync): add reproducible 26-task runner - #15

Draft
wpyaoxueros wants to merge 1 commit into
mainfrom
release/sync39-d9f83ac-public
Draft

eval(sync): add reproducible 26-task runner#15
wpyaoxueros wants to merge 1 commit into
mainfrom
release/sync39-d9f83ac-public

Conversation

@wpyaoxueros

Copy link
Copy Markdown
Collaborator

Summary

  • Adds the synchronous 26-task rollout runner used for the completed 520-episode evaluation.
  • Uses external, per-task VLM and VLA asset interfaces rather than private filesystem paths.
  • Locks official stage scoring to d9f83ac5182e25ad7f0a301a77a0b667f2392df1; missing scorer assets fail fast.

Validation

  • 110 passed under tests/sync_eval_26.
  • 26-profile preflight passed against the locked local asset map.
  • Public-path audit passed: no user-specific filesystem paths or account names in the sync runner/tests.

Remaining before review

This PR stays draft until the published Hugging Face weight repository has completed upload and all profiles are updated to its immutable revision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant