Skip to content

test(render): make automatic backend checks independent of timing - #1915

Open
TimotheusG wants to merge 2 commits into
storytold:mainfrom
TimotheusG:fix/auto-backend-test-timing
Open

TimotheusG wants to merge 2 commits into
storytold:mainfrom
TimotheusG:fix/auto-backend-test-timing

Conversation

@TimotheusG

@TimotheusG TimotheusG commented Oct 19, 2026 •

Copy link
Copy Markdown

The automatic-backend tests inferred CPU/GPU performance from real scheduling and allocation times. During full Windows CI, auto_backend_keeps_fast_accelerators failed with "asked 20 of 40" even though the mock had no requested delay. A zero-delay mock is not guaranteed to outperform the CPU under concurrent load.

The same timing failure was already reported in #1732, #1855 and #1874.

Feed the production AutoPick selector controlled timing samples to check warm-up, periodic re-probes, faster/tied/noisy GPU costs, and a clearly slower GPU. Keep actual renderer dispatch and pixel checks using the resulting known history, including explicit GPU and Software Only overrides. The declining-accelerator and changing-cost tests remain. Production selection code is unchanged; the mock no longer sleeps.

Validation: all four backend-selection tests pass, including ten repeat runs with 64 test threads. Full cargo xtask ci passed on Windows on the integration branch: 3,683 passed, 0 failed, 33 existing/opt-in ignored, strict Clippy, formatting, audits, and 32 WebAssembly crates. That branch combines #1889, #1890, #1891, #1827 and #1916 (Windows viewer screenshot guard); this PR changes only the backend-selection test file.

Summary by CodeRabbit

  • Tests
    • Updated rendering tests to verify automatic renderer selection using supplied CPU and GPU performance measurements.
    • Added checks for renderer behavior across varying GPU costs and when an accelerator is unavailable.
    • These changes affect test coverage only; no end-user-visible behavior is changed.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0f7f3ede-e360-4eed-8168-5e3b2c0c46be
📥 Commits

Reviewing files that changed from the base of the PR and between e551cbe and c4505d3.

📒 Files selected for processing: 1
  • crates/render/src/tests_auto.rs
🚧 Files skipped from review as they are similar to previous changes: 1
  • crates/render/src/tests_auto.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

📝 Walkthrough

The Auto renderer tests replace sleep-based timing with a mock accelerator and explicit CPU and GPU costs. They check Auto selection, explicit GPU selection, disabled GPU acceleration, and retries with a declining accelerator.

Changes: Auto renderer tests

Layer / File(s) Summary
Deterministic Auto selection tests
crates/render/src/tests_auto.rs
MockAccel replaces the sleeping test accelerator. measured_frames records supplied CPU or GPU costs. Tests check Auto selection at several cost values, explicit renderer behavior, and accelerator retry requests.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: echelon

Merge Risk: ⚪ Minimal · up to c4505

No actionable issue remains from this review; the PR is mergeable after normal checks.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: updating render tests so automatic backend checks do not depend on timing.
Docstring Coverage Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 1 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.98.1)

Clippy execution timed out

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant