Skip to content

feat: visual regression guard for the Pixi client #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
N1k4G merged 4 commits into from
Aug 24, 2026
Merged

feat: visual regression guard for the Pixi client #133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c605ad7
feat: visual regression guard for the Pixi client
N1k4G Aug 24, 2026
07f7d93
fix: key reference frames by platform, and seed the linux set from CI
N1k4G Aug 24, 2026
3f66729
chore: commit the linux reference set, drop the temporary seeding steps
N1k4G Aug 24, 2026
635fac4
fix: capture the torch state the scene claims, build before checking,…
N1k4G Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,20 @@ jobs:
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium

- name: Pixi visual regression check
run: npm run pixi:visual-check

# As in pr.yml: the breach diagnostic is written to disk and would
# otherwise die with the runner.
- name: Upload Pixi visual diff frames
if: failure()
uses: actions/upload-artifact@v7
with:
name: pixi-visual-actual-frames
path: tests/fixtures/reference-frames/pixi/**/*.actual.png
if-no-files-found: ignore
retention-days: 7

- name: Run migration client unit tests
run: npm run test:unit

Expand Down
16 changes: 16 additions & 0 deletions .github/workflows/pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,22 @@ jobs:
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium

- name: Pixi visual regression check
run: npm run pixi:visual-check

@N1k4G N1k4G Aug 24, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Upload the actual frames when this CI step fails

On a breach the checker writes tests/fixtures/reference-frames/pixi/linux/*.actual.png and tells the reviewer to inspect those files, but neither workflow uploads that path. The runner then exits and the only copy is lost; the existing always-upload steps cover screenshots/ and playwright-report/ only. This is especially blocking for a platform-keyed guard because a Windows maintainer cannot reproduce the Linux pixels locally. Please add an if: failure()/always() artifact step for the actual PNGs (in PR and preferably deploy workflows) so the promised diagnostic is retrievable.


# The checker writes <scene>.actual.png beside its reference on a breach
# and tells the reviewer to look at it. Without this the runner exits and
# the only copy is gone — and the references are platform-keyed, so a
# maintainer on another OS cannot reproduce these pixels locally.
- name: Upload Pixi visual diff frames
if: failure()
uses: actions/upload-artifact@v7
with:
name: pixi-visual-actual-frames
path: tests/fixtures/reference-frames/pixi/**/*.actual.png
if-no-files-found: ignore
retention-days: 7

- name: Run migration client unit tests
run: npm run test:unit

Expand Down
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,7 @@ screenshots/
# OS
.DS_Store
Thumbs.db

# Debug output from `npm run pixi:visual-check` when a scene breaches its
# budget. The references beside them are committed; these are not.
tests/fixtures/reference-frames/**/*.actual.png
6 changes: 5 additions & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@
"prepare": "husky",
"baseline:visual-compare": "node scripts/compare-rendering.mjs",
"sites:generate": "node scripts/generate-site-resources.mjs",

@N1k4G N1k4G Aug 24, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Make the visual commands build the client they are checking

Both advertised commands consume dist/, but neither npm script builds it; the checker only verifies that dist/index.html exists. Because dist is gitignored and persists locally, npm run pixi:visual-check can report a perfect 0% delta after a Pixi source change while testing an older build, and pixi:visual-update can record references from stale code. CI happens to run build first, but the documented standalone commands are not trustworthy. Please add build prehooks for both check and update (as test:e2e already does) or have the harness build/serve current sources itself.

"sites:check": "node scripts/generate-site-resources.mjs --check"
"sites:check": "node scripts/generate-site-resources.mjs --check",
"prepixi:visual-check": "npm run build",
"pixi:visual-check": "node scripts/pixi-visual-check.mjs",
"prepixi:visual-update": "npm run build",
"pixi:visual-update": "node scripts/pixi-visual-check.mjs --update"
},
"devDependencies": {
"@playwright/test": "^1.62.0",
Expand Down
Loading