diff --git a/README.md b/README.md index 0d21bcc0..6a5b41b8 100644 --- a/README.md +++ b/README.md @@ -97,22 +97,29 @@ Then run `bundle install`. **Defaults work for most Rails apps.** `blur_active_element`, `hide_caret`, and `fail_if_new` (in CI) are enabled automatically. -If you see inconsistent results, add tolerance: +If you see inconsistent results, choose a color comparison method: ```ruby +# Option 1: Perceptual (recommended, VIPS only) +Capybara::Screenshot::Diff.perceptual_threshold = 2.0 + +# Option 2: Tolerance-based comparison (legacy) +Capybara::Screenshot::Diff.tolerance = 0.0005 + +# Always set window_size for consistent dimensions Capybara::Screenshot::Diff.configure do |screenshot, diff| screenshot.window_size = [1280, 1024] - diff.tolerance = 0.0005 end ``` -| Use Case | VIPS `tolerance` | ChunkyPNG `color_distance_limit` | -|----------|-----------------|--------------------------------| -| Standard Rails apps | 0.0005 | 15 | -| Animated/complex pages | 0.01 | 30 | -| Pixel-perfect design | 0.0001 | 5 | +| Use Case | VIPS `perceptual_threshold` | VIPS `tolerance` | ChunkyPNG `color_distance_limit` | +|----------|---------------------------|-----------------|--------------------------------| +| Cross-OS/browser testing | 2.0 (recommended) | — | — | +| Standard Rails apps | — | 0.001 (default) | 15 | +| Animated/complex pages | — | 0.01 | 30 | +| Pixel-perfect design | — | 0.0001 | 5 | -See [Configuration Reference](docs/configuration.md) for all options. +**⚠️ Color methods are exclusive:** Use `perceptual_threshold` OR `color_distance_limit`, not both. But `tolerance` works with either — it's applied by default for VIPS (0.001). See [Choosing the Right Method](docs/configuration.md#choosing-the-right-color-comparison-method). ## Troubleshooting diff --git a/docs/configuration.md b/docs/configuration.md index e94e4501..e540ae08 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -26,9 +26,53 @@ end | Use Case | VIPS `tolerance` | ChunkyPNG `color_distance_limit` | `stability_time_limit` | |----------|-----------------|--------------------------------|----------------------| | Animated/complex pages | 0.01 | 30 | 2s | -| Standard Rails apps | 0.0005 | 15 | 1s | +| Standard Rails apps | 0.001 (default) | 15 | 1s | | Pixel-perfect design tests | 0.0001 | 5 | 1s | +**Note:** VIPS defaults to `tolerance: 0.001` (allows 0.1% pixel difference). ChunkyPNG has no default tolerance. + +## Choosing the Right Color Comparison Method + +**Important:** `perceptual_threshold`, `color_distance_limit`, and `tolerance` serve different purposes. Use this decision tree: + +### Step 1: Choose color comparison method (pick ONE) + +| Method | Scale | Driver | Best for | +|--------|-------|--------|----------| +| `perceptual_threshold` | 0-100+ (dE00) | VIPS only | Cross-OS/browser font rendering, anti-aliasing | +| `color_distance_limit` | 0-510 (RGBA Euclidean) | VIPS, ChunkyPNG | Legacy setups, fine-grained RGB control | + +**Recommendation:** Use `perceptual_threshold: 2.0` for most cases. It matches human perception and needs less tuning. + +**⚠️ Color comparison methods are exclusive:** `perceptual_threshold` and `color_distance_limit` cannot both be active — if you set both, `perceptual_threshold` wins and `color_distance_limit` is ignored. However, `tolerance` works with **both** methods and is applied by default for VIPS (0.001). This means even with `perceptual_threshold: 2.0`, the `tolerance: 0.001` default still filters results. + +### Step 2: Set tolerance (optional, independent) + +| Setting | What it does | Scale | +|---------|--------------|-------| +| `tolerance` | Maximum allowed *ratio* of different pixels (VIPS) or diff bounding box (ChunkyPNG) | 0.0-1.0 | + +**Example:** `tolerance: 0.001` allows 0.1% of the image to differ (e.g., 125 pixels in a 1280×1024 screenshot). + +**Key difference:** +- `perceptual_threshold` / `color_distance_limit` → **"how different can a pixel be?"** +- `tolerance` → **"how many pixels can differ?"** + +**⚠️ Driver difference:** VIPS counts actual different pixels. ChunkyPNG counts the bounding box area around differences — a single pixel diff creates a box, and the entire box area counts against tolerance. This makes ChunkyPNG stricter with the same tolerance value. + +### Quick start + +```ruby +# Modern approach (recommended) +screenshot 'dashboard', perceptual_threshold: 2.0 + +# Allow small noise regions +screenshot 'dashboard', perceptual_threshold: 2.0, tolerance: 0.001 + +# Legacy ChunkyPNG setup +screenshot 'dashboard', color_distance_limit: 15 +``` + ## Configuration Tiers **Tier 1 — Zero config (works immediately):** diff --git a/docs/drivers.md b/docs/drivers.md index 3c840332..ffd8765b 100644 --- a/docs/drivers.md +++ b/docs/drivers.md @@ -27,9 +27,16 @@ Capybara::Screenshot::Diff.perceptual_threshold = 2.0 Use `perceptual_threshold` when you see false positives from font rendering differences across CI environments, or when `color_distance_limit` with raw RGB requires frequent tuning. -**Note:** `perceptual_threshold` and `color_distance_limit` are independent options on different -scales. `perceptual_threshold` uses dE00 (0-100+), `color_distance_limit` uses Euclidean RGB -distance (0-441). Set one or the other, not both. +**⚠️ Important:** `perceptual_threshold` and `color_distance_limit` are **mutually exclusive**. +If you set both, `perceptual_threshold` takes priority and `color_distance_limit` is silently ignored. + +These options use different scales and algorithms: +- `perceptual_threshold` → CIE dE00 perceptual distance (0-100+) +- `color_distance_limit` → Euclidean RGBA distance (0-510) + +**Choose one based on your driver setup:** +- VIPS with `ruby-vips` gem → prefer `perceptual_threshold` +- ChunkyPNG (no native dependencies) → use `color_distance_limit` ## Available Image Processing Drivers @@ -65,14 +72,16 @@ You can use the `tolerance` option to the `screenshot` method to set level: ```ruby test 'unstable area' do visit '/' - screenshot 'index', tolerance: 0.3 + # tolerance: 0.01 allows 1% of pixels to differ (use for noisy pages) + screenshot 'index', tolerance: 0.01 end ``` You can also set this globally: ```ruby -Capybara::Screenshot::Diff.tolerance = 0.3 +# Default for VIPS is 0.001 (0.1% pixel difference allowed) +Capybara::Screenshot::Diff.tolerance = 0.001 ``` ## Median filter size (vips only) diff --git a/lib/capybara_screenshot_diff/dsl.rb b/lib/capybara_screenshot_diff/dsl.rb index a6552dc5..96db07fe 100644 --- a/lib/capybara_screenshot_diff/dsl.rb +++ b/lib/capybara_screenshot_diff/dsl.rb @@ -41,7 +41,11 @@ def screenshot_group(name) # @option options [Array] :crop [left, top, right, bottom] Edge coordinates to crop the screenshot to. # @option options [Array>] :skip_area Array of [left, top, right, bottom] edge coordinates to ignore. # @option options [Numeric] :tolerance (0.001 for :vips driver) Color tolerance for comparison. + # Represents the maximum allowed ratio of different pixels (0.0-1.0 scale). # @option options [Numeric] :color_distance_limit Maximum allowed color distance between pixels. + # Uses Euclidean RGBA distance (0-510 scale). Mutually exclusive with :perceptual_threshold. + # @option options [Numeric] :perceptual_threshold Maximum perceptual color difference (CIE dE00). + # Uses human perception-based scale (0-100+). VIPS only. Takes priority over :color_distance_limit if both set. # @option options [Numeric] :shift_distance_limit Maximum allowed shift distance for pixels. # @option options [Numeric] :area_size_limit Maximum allowed difference area size in pixels. # @option options [Symbol] :driver (:auto) The image processing driver to use (:auto, :chunky_png, :vips).