Add no-checkpoint adapter inspector and guide - #9
Open
09Catho wants to merge 1 commit into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a lightweight “no-checkpoint” inspection workflow for ACoT adapter/config debugging, so users can understand expected inputs/transforms/masks (and optionally validate a small sample) before running training, downloading assets, or serving policies.
Changes:
- Adds
scripts/inspect_acot_adapter.pyto build a JSON/text report of model/data/transform/mask expectations and optionally validate a.npzsample. - Adds
scripts/inspect_acot_adapter_test.pyto cover mask summarization, sample validation, and report JSON stability. - Adds
docs/acot_adapter_guide.mdplus a README link to document the workflow and key ACoT adapter concepts.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| scripts/inspect_acot_adapter.py | New inspector tool to summarize config expectations without checkpoints and optionally validate a sample. |
| scripts/inspect_acot_adapter_test.py | Unit tests for inspector report/mask/sample validation behavior. |
| docs/acot_adapter_guide.md | New guide for adapter inspection/adaptation, masks, and remote serving flow. |
| README.md | Adds an earlier link to the new guide and inspector usage example. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+107
to
+112
| observation = { | ||
| "observation/image": image_tools.convert_to_uint8(image_tools.resize_with_pad(base_img, 224, 224)), | ||
| "observation/wrist_image": image_tools.convert_to_uint8(image_tools.resize_with_pad(wrist_img, 224, 224)), | ||
| "observation/state": state, | ||
| "prompt": task_instruction, | ||
| } |
| Before adapting a new LeRobot dataset or robot client, see the [ACoT adapter guide](docs/acot_adapter_guide.md). You can inspect a config without checkpoints or GPUs: | ||
|
|
||
| ```bash | ||
| uv run python scripts/inspect_acot_adapter.py --config-name <CONFIG_NAME> |
Comment on lines
+289
to
+305
| data_factory = train_config.data | ||
| original_load_norm_stats = getattr(data_factory, "_load_norm_stats", None) | ||
| if original_load_norm_stats is None: | ||
| return None, first_error | ||
|
|
||
| def _skip_norm_stats(*_: Any, **__: Any) -> None: | ||
| return None | ||
|
|
||
| try: | ||
| object.__setattr__(data_factory, "_load_norm_stats", _skip_norm_stats) | ||
| data_config = data_factory.create(train_config.assets_dirs, train_config.model) | ||
| except Exception as exc: | ||
| return None, f"{first_error}; fallback without norm stats also failed with {type(exc).__name__}: {exc}" | ||
| finally: | ||
| object.__setattr__(data_factory, "_load_norm_stats", original_load_norm_stats) | ||
|
|
||
| return data_config, f"{first_error}; inspected transforms with norm-stat loading disabled." |
Comment on lines
+14
to
+16
| ```bash | ||
| uv run python scripts/inspect_acot_adapter.py --config-name acot_icra_simulation_challenge_reasoning_to_action | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi maintainers,
@LinqingZhong
I was reading the ACoT-VLA paper and trying to understand how someone would practically adapt this repo to their own LeRobot-style dataset or robot setup. While going through the configs, I found that a lot of useful information is already present in the code, but it is hard for a new user to see it without first setting up checkpoints, assets, or a full training run.
This PR adds a small no-checkpoint inspection tool and a guide for that workflow.
What this adds
scripts/inspect_acot_adapter.pystate_mask,action_mask, anddelta_action_mask.npzsample without running inferencedocs/acot_adapter_guide.mdA README link near the training/inference section so users can discover the guide earlier.
Why I think this is useful
This should help people debug adapter/config issues before they try expensive training or policy serving. It is especially useful while official checkpoints are still pending, because users can still inspect the public configs and understand what the repo expects.
It also relates to a few existing community questions:
Notes
The tool is intentionally read-only. It does not load model weights, start a policy server, run inference, download datasets, or change training behavior.
For configs whose private norm-stat paths are not available locally, the script reports that warning and still inspects the transform chain by retrying with norm-stat loading disabled. This is only for inspection and does not modify any training code path.
Validation
I tested the focused changes locally and in WSL CPU mode.
Passed:
pytest scripts src/openpi/transforms_test.py -m "not manual" -q ruff check scripts/inspect_acot_adapter.py ruff format --check scripts/inspect_acot_adapter.py python scripts/inspect_acot_adapter.py --config-name acot_icra_simulation_challenge_reasoning_to_actionThe inspector prints the expected ACoT details for the ICRA challenge config, including:
acot_pi05303032top_head,hand_left,hand_right