voice-vr

A microphone-driven 3D experiment built with React and A-Frame. A smoothed loudness history drives traveling ripples across a sphere.

Open the demo · Build and deployment status

Run locally

Use Node 22.12 or later and npm:

npm ci
npm run dev

Open the localhost URL printed by Vite and click Start microphone in the settings panel. Allow microphone access. Click Stop microphone to release it. The scene's gaze-cursor control also works. Permission errors appear in the panel. Microphone access requires localhost or HTTPS. Audio stays in the browser; this application does not upload or record it.

Ripple controls

All controls apply live without restarting the microphone. Settings last until the page reloads; Reset settings restores the defaults.

Control	Meaning
Ripple origin	Bottom to top, top to bottom, or center to both poles. Center means the equator, not the interior of the sphere.
Travel time	0.25–8 seconds for sound to reach the farthest point from the origin. Longer times show more history across the surface.
Input smoothing	0–300 ms of exponential smoothing on the RMS loudness envelope. Larger values soften transients and introduce more response lag.
Ripple strength	Maximum outward displacement, from 0 to 1.5 scene units. Zero disables deformation.
Input gain	Visual sensitivity from 1× to 16×, with soft saturation. This does not change the microphone's recording gain.
Mesh detail	Low, medium, or high sphere resolution. Higher resolution helps narrow ripples, at a higher GPU cost.

Loudness is measured in 128-sample windows, independent of rendering speed. A timestamped ten-second history is resampled into a linearly filtered texture every animation frame. Silence maps to zero displacement instead of inflating the shape. Changing smoothing affects incoming sound; existing history retains its previous smoothing. Stopping clears the history.

The HTML settings panel is available on desktop and mobile. Set parameters before entering immersive VR; an in-headset settings panel is not implemented.

npm run lint
npm run format:check
npm run build
npm run preview

The build goes into the ignored public directory. npm uses package-lock.json; the old Yarn lockfile and Parcel setup have been replaced.

GitHub Pages

Site URL: francois-esquire.github.io/voice-vr.

The Build and deploy workflow runs lint, formatting, and the production build for pull requests and pushes to main. Successful pushes deploy the built public artifact to GitHub Pages in a separate job. Pull requests never deploy. You can also run the workflow manually from main.

Pages uses GitHub Actions as its deployment source. Vite's base path is /voice-vr/ so asset URLs work under the repository's Pages URL. No npm publication, deployment secrets, test suite, or coverage gate is involved.

Audio pipeline

MediaStreamSourceNode → ScriptProcessorNode → smoothed RMS history → canvas texture → DataTexture → shader

ScriptProcessorNode is deliberately retained for this pass, despite its deprecation. AudioWorklet and alternative audio sources, including generated voice or combined inputs, are future work. There is no speech recognition or voice AI integration yet.

React stays on 16.14 to limit API changes. A-Frame moves to 1.8 because 0.9 calls the removed navigator.xr.requestDevice API in current browsers. The shader sampler is named audioTexture to avoid the WebGL2 texture function name. Three.js comes from A-Frame rather than a separate dependency.

Manual checks

• Start, stop, and restart streaming. The shape should react to sound and the microphone indicator should turn off after stopping.
• Deny microphone permission and check the error message.
• Leave the page while permission is pending or streaming is active; microphone tracks must not remain active.
• Check the browser console for shader errors.

There is no committed automated test suite. Desktop browser checks do not establish headset compatibility; test on a physical device before relying on immersive VR.

Background

This project began as a learning experiment for an A-Frame and React tutorial. That article describes the original version; use the instructions above for the current app.