Refracta: real-time refractive vision simulator (Astigmastism and Myopia sim)

Try here: https://sane24.github.io/Refractive-Vision-Simulator/

What the world looks like through your prescription Refracta renders any scene, live webcam, uploaded photo, or one of ten bundled environments, the way it lands on the retina of an eye with myopia, hyperopia, astigmatism, or presbyopia. The blur is not a Photoshop gaussian: it is the geometric point-spread function of a sphero-cylindrical eye, oriented by the prescription's AXIS, scaled by pupil diameter, and driven per-pixel by scene depth and the eye's accommodation state. A Python/OpenCV reference implementation is the ground truth; the GLSL shader that runs at 60 fps is validated against it to 0.006/255 RMS on this page: open the demo with ?validate=1 and watch it re-prove itself in your browser.

Split-wipe on the night-drive scene, then an AXIS sweep: −1.00 −2.75 × θ, 6 mm pupil. Every light source smears along the axis+90 meridian, and rotating AXIS visibly rotates the smear, which is the visual signature no isotropic blur can fake.

• 10 switchable environments with exact, procedurally generated depth maps: night drive 🌃, traffic lights 🚦, Snellen chart 👁️, restaurant menu 🍝, highway billboard 🛣️, classroom 🧑‍🏫, phone at 30 cm 📱, starry night ✨, grocery shelf 🛒, sunny park 🌳 (+ an engineering test card 🎯).
• Live webcam (graceful fallback if denied) and photo upload, with optional depth: upload a depth map, run tools/depth.py offline, or click "Estimate depth with AI" to run Depth-Anything-V2 in the browser.
• Preset prescriptions (mild/high myopia, hyperopia, with-the-rule and oblique astigmatism, presbyopia, dilated night driver) plus full sliders: SPH, CYL, AXIS, ADD, pupil, focus target, FOV, chromatic aberration.
• Split-wipe comparison with a draggable divider, live clinical notation (−2.00 −1.50 × 090), an estimated uncorrected acuity readout, and a live drawing of the current PSF ellipse.

Optics physics

1. Defocus. A refractive error of ΔD diopters through a pupil of diameter p meters spreads a point over a blur disk of angular diameter

β ≈ p · ΔD                      [radians]
blur_px = β / rad_per_pixel,    rad_per_pixel = 2·tan(FOV/2) / width_px

Intuition: the pupil is the aperture of an out-of-focus camera; more diopters or a bigger pupil both widen the cone of confusion. This is why the same −2.00 D feels far worse at night: the dark-adapted pupil doubles p.

2. Astigmatism: the oriented PSF. A sphero-cylindrical eye has two focal powers on perpendicular meridians: SPH along AXIS and SPH + CYL along AXIS+90. A point source therefore images as an ellipse (a cross-section of Sturm's conoid): the pupil disk scaled per-meridian by each meridian's residual defocus,

r_axis  ∝ |V + SPH − A|         (semi-axis along AXIS)
r_cross ∝ |V + SPH + CYL − A|   (semi-axis along AXIS+90)

so a streak, not a halo. Sanity check that the code enforces: a pure cylinder with a horizontal axis has all its error in the vertical meridian, so points smear vertically.

3. Depth and accommodation. Objects live at vergence V = 1/distance. The simulated eye accommodates A diopters (clamped to the reserve implied by the ADD: A_max = 2.5 − ADD) to put the circle of least confusion on the retina for your chosen focus target. Per pixel, the depth texture supplies V, and ΔD = V + SPH − A per meridian: the focal plane snaps sharp while everything nearer and farther blurs by its true dioptric distance. Depth maps store normalized disparity (linear in 1/d), so defocus is linear in the stored value and 8 bits are plenty.

4. Emissive HDR expansion. 8-bit sources clip a headlight at 1.0, and blurring clipped pixels yields dim grey smudges. Near-white pixels are expanded back toward plausible radiance before convolution (inverse tonemapping), then the result is re-clamped: streaks stay vivid, exactly like real night glare. With the boost at 0 the pipeline is exactly linearize → convolve → encode, which is what validation uses.

5. Chromatic aberration (optional). The eye carries ~1.2 D of longitudinal chromatic aberration across the visible band; blue focuses in front of the retina. Modeled as per-channel defocus offsets (R +0.35 / G 0 / B −0.85 D), it fringes night lights in a very recognizable way.

Architecture

flowchart LR
    subgraph "Python ground truth"
        P[reference/psf.py<br>PSF + vergence math<br>12 unit tests] --> S[reference/simulate.py<br>OpenCV convolution]
        P --> A[reference/validate_acuity.py<br>tumbling-E ideal observer]
    end
    subgraph "content pipeline"
        G[tools/make_scenes.py<br>10 environments +<br>exact disparity maps] --> W
        D[tools/depth.py<br>Depth-Anything-V2 / MiDaS<br>for user photos] --> W
    end
    subgraph "web app (static)"
        W[js/optics.js<br>same formulas → uniforms] --> F[shaders/vision.frag<br>oriented variable-radius<br>gather at 60 fps]
        F --> UI[split wipe · presets ·<br>webcam · PSF inset]
    end
    S -->|reference PNGs| V[?validate=1<br>in-browser diff:<br>RMS / PSNR table]
    F --> V

Loading

The same formulas exist exactly three times, on purpose: NumPy (tested, readable), JS (bakes uniforms), GLSL (per-pixel). The validation page is the proof they never drift.

Inside shaders/vision.frag the transport is split, and the comments walk through it line by line: pass A gathers over the center pixel's ellipse (a golden-angle spiral through the ellipse matrix, mip-biased so large kernels stay noise-free), which is the classic depth-of-field gather and correct for surfaces. Pass B handles emissive energy (anything HDR-expanded above 1.0) with scatter-as-gather: each tap asks "does your PSF, computed from your depth, reach me?", Monte-Carlo normalized, so a headlight's streak correctly paints over sharp neighbors, something a center-kernel gather cannot do. An exact mode (dense texelFetch loop, border-replicate) reproduces the Python convolution for validation and stills.

Validation

Shader vs reference. Open ?validate=1: each case renders the shader in exact mode on the test card and diffs it against the committed OpenCV output. Current results (Apple M-series, Chrome):

case	prescription	RMS /255	PSNR
identity	plano (must be lossless)	0.000	168.1 dB
myopia	−4.00 sph, pupil 4 mm	0.003	97.3 dB
astig-30	−1.00 −2.50 × 030, pupil 5 mm	0.002	101.6 dB
astig-117	plano −3.00 × 117, pupil 6 mm	0.001	111.6 dB
presby-near	+1.50 ADD 2.50 at 50 cm	0.006	92.4 dB

Diopters → acuity. reference/validate_acuity.py builds tumbling-E optotypes at exact Sloan proportions, blurs them with the reference PSF (4 mm pupil), and asks an ideal observer to tell the four orientations apart from the blurred templates. The discrimination threshold is calibrated once to the classic clinical anchor 2.00 D ≈ 20/200; every other row is then a prediction:

defocus	blur disk	simulated acuity	clinical rule of thumb
0.00 D	0.0′	20/15	20/20 (display-limited)
0.25 D	3.4′	20/20	20/20 – 20/25
0.50 D	6.9′	20/40	20/40
0.75 D	10.3′	20/70	20/50 – 20/70
1.00 D	13.8′	20/100	20/70 – 20/150
1.50 D	20.6′	20/200	~20/200
2.00 D	27.5′	20/200	20/200 (anchor)
3.00 D	41.2′	20/300	20/300+

The resolvable denominator grows linearly with defocus (R² = 0.95), which is the physical claim under test: MAR ∝ blur-disk diameter ∝ p · ΔD.

Depth estimation. Bundled scenes carry exact procedural depth. For user photos, tools/depth.py runs Depth-Anything-V2 (or MiDaS via torch.hub); on the synthetic park scene the estimate correlates with ground truth at Pearson r = 0.61, a fair illustration of why relative monocular depth needs the app's near/far calibration sliders:

Gallery (raw shader output)

cyl axis 0: vertical streaks	same cyl at axis 90
stars render the PSF itself (LCA on)	presbyopia: menu at 42 cm unreadable, candles bokeh
accommodating at the board: notes blur	accommodating at the notes: board blurs

Repo layout

index.html  js/  css/  shaders/     the app (static, no build step)
shaders/vision.frag                 the heavily-commented core
reference/  psf.py  simulate.py     Python ground truth + unit tests
            validate_acuity.py      diopter→acuity experiment
tools/      make_scenes.py          procedural environments + exact depth
            depth.py  serve.py  make_validation_refs.py  make_gif.py
assets/scenes/                      10 environments (+ test card) with depth
validation/ cases.json  refs/       inputs for the in-browser validation

Current limitations

• Geometric PSF only. The kernel is the uniform pupil-projection ellipse; diffraction, Stiles-Crawford apodization, higher-order aberrations (coma, spherical) and intraocular scatter are not modeled, so tiny blurs (< ~0.25 D) are slightly too clean and real night halos have more texture.
• Relative depth for photos. Monocular networks return ordering, not meters; the near/far sliders are an explicit, honest calibration step. Bundled scenes sidestep this with exact synthetic depth.
• Monocular, static accommodation. One eye, no binocular summation or rivalry; accommodation is a least-confusion policy with an age-derived amplitude (2.5 − ADD), not a dynamic control loop, and night myopia / accommodative lead-lag are ignored.
• Scatter approximations in fast mode. Pass A uses the center pixel's kernel (mild halos at hard depth edges); pass B is Monte-Carlo with a clamped estimator (very sharp emitters route through A). Exact mode exists precisely so these shortcuts are measurable: they never touch validation.
• Display ≠ retina. The screen has its own MTF, and 20/15 is the finest row the bundled chart's pixel grid can encode at its 15° FOV.

Stack

piece	role
Python + NumPy + OpenCV	reference optics, unit tests, scene/asset pipeline
Depth-Anything-V2 / MiDaS	monocular depth for user photos (offline + in-browser)
Three.js (vendored) + WebGL2	texture plumbing, render loop, webcam
GLSL ES 3.0	vision.frag: the oriented variable-radius PSF, exact + fast paths
Vanilla JS/HTML/CSS	app shell; ships as a static site

Refracta is an educational visualization, not a medical device; it will not refract your eyes.