Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/experiment_result.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ body:
id: wamprobe_revision
attributes:
label: WAMProbe version and commit
placeholder: 0.1.0rc1 / full Git commit SHA
placeholder: 0.1.0 / full Git commit SHA
validations:
required: true
- type: textarea
Expand Down
10 changes: 5 additions & 5 deletions .github/ISSUE_TEMPLATE/external_reproduction.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
name: External reproduction
description: Report an independent clean-install WAMProbe smoke reproduction
name: Optional external reproduction
description: Share an independent clean-install WAMProbe smoke reproduction
title: "[External reproduction]: "
labels: ["reproduction"]
body:
- type: markdown
attributes:
value: >-
Thank you for independently checking WAMProbe. This form records the
evidence needed for the external-smoke release gate. Failed attempts are
useful too. Do not post credentials or private machine paths.
Thank you for independently checking WAMProbe. External reports are useful
additional evidence but are not a v0.1 release gate. Failed attempts are useful
too. Do not post credentials or private machine paths.
- type: input
id: reproducer
attributes:
Expand Down
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,31 @@

All notable changes to WAMProbe will be documented here.

## [0.1.0] - 2026-07-16

This stable release promotes the audited CPU core and real-model evaluation slice after
repeated maintainer clean-install checks from both GitHub artifacts and public PyPI. The
evidence is recorded in
[Issue #2](https://github.com/myheart521/WAMProbe/issues/2). These owner-run checks are
maintainer acceptance, not independent external reproduction.

### Added

- English and complete Simplified Chinese READMEs with generated project diagrams and an
actual PointMass report preview.
- A tested custom state-future adapter starter that writes the standard JSON, JSONL,
Markdown, and HTML report contract.
- A detailed adapter integration guide, dependency-aware roadmap, structured result and
reproduction forms, repository Topics, and labeled contribution-sized Issues.

### Changed

- Promoted package, citation, model-card, evidence-manifest, and documentation metadata to
stable `0.1.0`.
- Included project documentation and the adapter starter in the source distribution.
- Made optional external reproduction distinct from the documented maintainer package
acceptance used for this release.

## [0.1.0rc1] - 2026-07-15

This candidate was tagged and published as a
Expand Down
3 changes: 2 additions & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ cff-version: 1.2.0
message: "If you use WAMProbe, please cite the software release."
title: "WAMProbe: Counterfactual Evaluation for World Action Models"
type: software
version: 0.1.0rc1
version: 0.1.0
date-released: "2026-07-16"
license: Apache-2.0
repository-code: "https://github.com/myheart521/WAMProbe"
authors:
Expand Down
5 changes: 3 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ Thank you for helping make WAM evaluation more causal, reproducible, and control
- **Share a run:** use the
[Experiment result](https://github.com/myheart521/WAMProbe/issues/new?template=experiment_result.yml)
form with standard artifacts and hashes.
- **Reproduce independently:** use the
- **Share an optional independent reproduction:** use the
[External reproduction](https://github.com/myheart521/WAMProbe/issues/new?template=external_reproduction.yml)
form. A maintainer-controlled run cannot satisfy the external-user gate.
form. A maintainer-controlled run must be labeled as maintainer evidence, never as an
independent report.
- **Pick a bounded task:** see the
[roadmap](docs/NEXT_STEPS.md) and issues labeled
[`good first issue`](https://github.com/myheart521/WAMProbe/labels/good%20first%20issue).
Expand Down
28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@
<a href="CONTRIBUTING.md"><strong>Contributing</strong></a>
</p>

> **Release status:** [`v0.1.0rc1`](https://github.com/myheart521/WAMProbe/releases/tag/v0.1.0rc1)
> is published on [PyPI](https://pypi.org/project/wamprobe/0.1.0rc1/) as a release
> candidate. The dependency-free CPU core is ready to use; the external reproduction
> report remains open in [Issue #2](https://github.com/myheart521/WAMProbe/issues/2).
> **Release status:** [`v0.1.0`](https://github.com/myheart521/WAMProbe/releases/tag/v0.1.0)
> is the stable release on [PyPI](https://pypi.org/project/wamprobe/0.1.0/).
> The dependency-free CPU core and maintainer clean-install acceptance are complete.
> [Issue #2](https://github.com/myheart521/WAMProbe/issues/2) records that evidence;
> no independent external reproduction is claimed.

## The problem WAMProbe tests

Expand Down Expand Up @@ -87,7 +88,7 @@ runs/pointmass-demo/
For an exactly pinned installation:

```bash
python -m pip install wamprobe==0.1.0rc1
python -m pip install wamprobe==0.1.0
```

## Read the result
Expand Down Expand Up @@ -291,7 +292,7 @@ Review the [reproducibility guide](docs/reproducibility/REPRODUCIBILITY.md) and
| Interpret a score correctly | [Core metric cards](docs/metrics/CORE_METRICS.md) |
| Add a model integration | [Custom adapter starter guide](docs/adapters/CUSTOM_ADAPTER.md) |
| Submit a structured experiment | [Experiment result form](https://github.com/myheart521/WAMProbe/issues/new?template=experiment_result.yml) |
| Reproduce independently | [External reproduction form](https://github.com/myheart521/WAMProbe/issues/new?template=external_reproduction.yml) |
| Share an optional independent reproduction | [External reproduction form](https://github.com/myheart521/WAMProbe/issues/new?template=external_reproduction.yml) |
| Reproduce LIBERO-CF-Mini | [Benchmark card](docs/benchmarks/LIBERO_CF_MINI.md) |
| Inspect the closed-loop protocol | [Experiment card](docs/experiments/TOY_CLOSED_LOOP_V0.1.md) |
| Audit the release | [Reproducibility guide](docs/reproducibility/REPRODUCIBILITY.md) |
Expand All @@ -301,17 +302,18 @@ Review the [reproducibility guide](docs/reproducibility/REPRODUCIBILITY.md) and

## Project status and roadmap

The `v0.1.0rc1` engineering scope is implemented and published. The remaining formal
acceptance item is one genuine independent clean-install smoke report in
[Issue #2](https://github.com/myheart521/WAMProbe/issues/2).
The `v0.1.0` engineering scope and maintainer clean-install acceptance are complete.
[Issue #2](https://github.com/myheart521/WAMProbe/issues/2) records owner-run checks from
both the public wheel and PyPI. These checks are intentionally described as maintainer
evidence, not independent external reproduction.

The [dependency-aware roadmap](docs/NEXT_STEPS.md) tracks evidence gates separately from
repository work. Next research milestones:
The [dependency-aware roadmap](docs/NEXT_STEPS.md) separates completed release gates from
the next research milestones:

1. expand LIBERO initial-state coverage;
2. evaluate action-conditioned real-WAM futures when an adapter exposes that capability;
3. add the Occluded-Object memory diagnostic to the broader toy tier;
4. review external reproduction evidence and promote a final `v0.1.0` release.
4. archive the stable release with a DOI and collect optional third-party reports.

## Contributing

Expand Down Expand Up @@ -346,7 +348,7 @@ If WAMProbe supports your work, cite the software release using
title = {WAMProbe: Counterfactual Evaluation for World Action Models},
author = {{WAMProbe contributors}},
year = {2026},
version = {0.1.0rc1},
version = {0.1.0},
url = {https://github.com/myheart521/WAMProbe}
}
```
Expand Down
26 changes: 14 additions & 12 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@
<a href="CONTRIBUTING.md"><strong>参与贡献</strong></a>
</p>

> **发行状态:**[`v0.1.0rc1`](https://github.com/myheart521/WAMProbe/releases/tag/v0.1.0rc1)
> 已作为候选版本发布到 [PyPI](https://pypi.org/project/wamprobe/0.1.0rc1/)。
> 无依赖的 CPU 核心已经可以直接使用;目前仍在
> [Issue #2](https://github.com/myheart521/WAMProbe/issues/2) 等待独立外部用户复现报告。
> **发行状态:**[`v0.1.0`](https://github.com/myheart521/WAMProbe/releases/tag/v0.1.0)
> 是发布在 [PyPI](https://pypi.org/project/wamprobe/0.1.0/) 上的正式稳定版。
> 无依赖 CPU 核心和维护者干净环境验收均已完成;
> [Issue #2](https://github.com/myheart521/WAMProbe/issues/2) 保存了验收证据,但项目不把它
> 描述成独立外部复现。

## WAMProbe 要解决的问题

Expand Down Expand Up @@ -82,10 +83,10 @@ runs/pointmass-demo/
└── report.html # 无需服务器即可打开的独立报告
```

如果需要固定到完全一致的候选版本
如果需要固定到完全一致的稳定版本

```bash
python -m pip install wamprobe==0.1.0rc1
python -m pip install wamprobe==0.1.0
```

## 查看结果
Expand Down Expand Up @@ -282,7 +283,7 @@ StarWAM 和 LIBERO 的隔离环境、GPU 预检、预处理来源及 smoke 命
| 正确解释指标 | [核心指标卡](docs/metrics/CORE_METRICS.md) |
| 接入一个新模型 | [自定义适配器接入指南](docs/adapters/CUSTOM_ADAPTER.md) |
| 提交结构化实验结果 | [实验结果表单](https://github.com/myheart521/WAMProbe/issues/new?template=experiment_result.yml) |
| 独立复现项目 | [外部复现表单](https://github.com/myheart521/WAMProbe/issues/new?template=external_reproduction.yml) |
| 提交可选的独立复现 | [外部复现表单](https://github.com/myheart521/WAMProbe/issues/new?template=external_reproduction.yml) |
| 复现 LIBERO-CF-Mini | [基准卡](docs/benchmarks/LIBERO_CF_MINI.md) |
| 检查闭环实验协议 | [实验卡](docs/experiments/TOY_CLOSED_LOOP_V0.1.md) |
| 审计发行版 | [可复现性指南](docs/reproducibility/REPRODUCIBILITY.md) |
Expand All @@ -292,15 +293,16 @@ StarWAM 和 LIBERO 的隔离环境、GPU 预检、预处理来源及 smoke 命

## 项目状态与路线图

`v0.1.0rc1` 工程范围已经实现并发布。正式验收目前只剩一项:由真实独立用户在
[Issue #2](https://github.com/myheart521/WAMProbe/issues/2) 提交干净环境安装复现报告。
`v0.1.0` 工程范围和维护者干净环境验收均已完成。
[Issue #2](https://github.com/myheart521/WAMProbe/issues/2) 保存了从公开 wheel 和 PyPI
执行的维护者验收记录。这些记录不会被称为独立外部复现。

[后续路线图](docs/NEXT_STEPS.md)会把外部证据门槛和仓库内工程任务分开跟踪。下一阶段研究计划
[后续路线图](docs/NEXT_STEPS.md)会把已完成的发布验收与后续研究任务分开跟踪

1. 扩展 LIBERO 初始状态覆盖;
2. 在适配器真正暴露对应能力后,评测动作条件真实 WAM 未来;
3. 在更完整的 Toy 层加入“遮挡物体记忆”诊断;
4. 审查外部复现证据,并发布正式 `v0.1.0`
4. 为稳定版建立 DOI 归档,并接收可选的第三方复现报告

## 参与贡献

Expand Down Expand Up @@ -334,7 +336,7 @@ pytest --cov=wamprobe --cov-report=term-missing --cov-fail-under=85
title = {WAMProbe: Counterfactual Evaluation for World Action Models},
author = {{WAMProbe contributors}},
year = {2026},
version = {0.1.0rc1},
version = {0.1.0},
url = {https://github.com/myheart521/WAMProbe}
}
```
Expand Down
49 changes: 26 additions & 23 deletions docs/NEXT_STEPS.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Next steps / 后续路线图

This roadmap separates work that can be automated in the repository from gates
that require independent people or new scientific evidence. A checked code task
must never be used as a substitute for an unmet evidence gate.
This roadmap separates completed release engineering from work that still needs new
scientific evidence. Maintainer verification and independent reproduction remain distinct
claims even though independent reproduction is not a `v0.1` release gate.

本路线图区分“仓库内可以自动完成的工程任务”和“必须由独立用户或新实验提供证据的验收
门槛”。代码完成不代表外部证据已经满足
本路线图区分已经完成的发布工程与仍需新科学证据的工作。维护者验收和独立复现始终是两种
不同声明,但独立复现不再作为 `v0.1` 的发布门槛

## Dependency path / 依赖关系

```text
P0 independent reproduction ──► P1 final v0.1.0
P0 maintainer clean install ──► P1 final v0.1.0

optional independent reports ──► additional public evidence

P2 contributor readiness ──► P3 broader paired benchmarks
└──► P4 action-conditioned real WAM
Expand All @@ -22,44 +24,45 @@ P1 stable release + public evidence ──► P5 DOI and outreach

| Priority | Milestone | Current state | Completion evidence |
|---|---|---|---|
| P0 | Independent clean-install reproduction / 独立干净环境复现 | **Blocked by an independent user** | A non-maintainer posts commit, environment, commands, exit status, elapsed time, and output hash in Issue #2 |
| P1 | Promote `v0.1.0rc1` to `v0.1.0` / 发布正式版 | **Blocked by P0** | P0 accepted, release audit green, signed/tagged release artifacts and provenance available |
| P2 | Contributor readiness / 贡献者接入能力 | **In progress** | Runnable adapter starter, contribution/result forms, roadmap, labeled starter issues, green CI |
| P0 | Maintainer clean-install acceptance / 维护者干净环境验收 | **Complete** | Public wheel and PyPI installs, exact commands, exit status, elapsed time, and output hash recorded in Issue #2 |
| P1 | Stable `v0.1.0` / 正式稳定版 | **Complete** | Release audit green, tagged artifacts and provenance available, published-package smoke passed |
| P2 | Contributor readiness / 贡献者接入能力 | **Complete** | Runnable adapter starter, contribution/result forms, roadmap, labeled starter issues, green CI |
| P3 | Broader paired diagnostics / 扩展配对诊断 | **Planned** | More LIBERO initial states plus Occluded-Object memory benchmark with restore/order checks |
| P4 | Real action-conditioned WAM evidence / 真实动作条件 WAM | **Planned; model capability required** | Public model exposes candidate-conditioned futures, adapter passes semantic tests, bounded report is reproducible |
| P5 | Archival and outreach / 归档与传播 | **After stable release** | Zenodo DOI, updated citation metadata, release notes, public result call |
| P5 | Archival and outreach / 归档与传播 | **Ready after v0.1.0** | Zenodo DOI, updated citation metadata, release notes, public result call |

Tracked work:

- P0: [independent reproduction gate #2](https://github.com/myheart521/WAMProbe/issues/2)
- P0: [maintainer acceptance record #2](https://github.com/myheart521/WAMProbe/issues/2)
- P3: [LIBERO initial-state coverage #10](https://github.com/myheart521/WAMProbe/issues/10)
and [Occluded-Object RFC #7](https://github.com/myheart521/WAMProbe/issues/7)
- P4: [public action-conditioned WAM audit #9](https://github.com/myheart521/WAMProbe/issues/9)
- Starter contributions: [adapter troubleshooting #11](https://github.com/myheart521/WAMProbe/issues/11)
and [malformed-output tests #12](https://github.com/myheart521/WAMProbe/issues/12)
- P5: [Zenodo DOI preparation #8](https://github.com/myheart521/WAMProbe/issues/8)

## P0 — independent reproduction
## P0 — maintainer clean-install acceptance

The maintainer can improve instructions and debug a failed report, but cannot
perform this acceptance item on the maintainer's own machine. The external user
should use the **External reproduction** issue form. Issue #2 remains the source
of truth.
The maintainer installed the public package in fresh environments, ran the CLI and bounded
demo, and recorded the version, environment, command, elapsed time, and output hash in
Issue #2. This completes the `v0.1` package acceptance chosen by the project owner. It does
not establish independent external reproduction.

维护者可以完善说明、协助排错,但不能在自己的机器上代替外部用户完成验收。真实独立用户
应通过 **External reproduction** 表单提交,Issue #2 是唯一验收记录。
维护者已经在全新环境安装公开发行包,运行 CLI 和有界 demo,并在 Issue #2 中记录版本、
环境、命令、耗时和输出哈希。这满足项目所有者为 `v0.1` 选择的包验收口径,但不代表独立
外部复现。

## P1 — stable release gate

After P0 is accepted:
The stable release procedure is:

1. confirm the candidate commit is clean and all required CI/security checks pass;
2. run the documented reproducible build and archive audit;
3. update version, changelog, citation, and release notes consistently;
4. publish the final tag and artifacts through the release workflow;
5. verify a clean install from the published package.

Do not publish `v0.1.0` before P0.
P0 is complete. Published-package smoke evidence closes P1.

## P2 — contributor readiness

Expand Down Expand Up @@ -100,12 +103,12 @@ after the semantic smoke test passes.
## P5 — DOI and outreach

Prepare Zenodo metadata and communication material after the stable release so
the DOI points to immutable public artifacts. Outreach should ask for independent
reproductions and comparable structured reports, not only stars or screenshots.
the DOI points to immutable public artifacts. The project may accept optional independent
reproductions and comparable structured reports without treating them as a release gate.

## Maintainer decision rules / 维护规则

- Never mark P0 complete from a maintainer-controlled machine.
- Never describe a maintainer-controlled check as independent reproduction.
- Never release a metric without an anti-gaming or failure-baseline test.
- Never label a proxy as a capability the model does not expose.
- Never commit checkpoints, datasets, credentials, or generated private evidence.
Expand Down
Loading