Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .github/docs/audio-reactivity-patterns.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Audio Reactivity Patterns for Visualizers

Lessons learned from fixing time-dependent audio drift in shader-based visualizers.

## The Core Problem: Audio × Time Drift

When audio-modulated values are **multiplied by elapsed time**, the visual impact of the same audio input grows unboundedly as the session continues.

```glsl
// BAD: audio-modulated speed × growing time = drift
float t = uTime * uSpeed; // uTime grows forever
rotation = rot(t * uTwistSpeed); // same audio fluctuation has bigger effect at t=600s vs t=10s
```

## The Fix: Phase Accumulators

Accumulate phase on the JS side. Audio modulates the **rate of change per frame**, not a multiplier on total elapsed time.

```typescript
// GOOD: accumulate phase incrementally
globalPhase += delta * currentSpeed; // only this frame's contribution
twistPhase += delta * currentSpeed * twistRate * audioBoost;

uniforms.uPhase.value = globalPhase;
uniforms.uTwistPhase.value = twistPhase;
```

```glsl
// Shader uses pre-accumulated phase directly
rotation = rot(uPhase * rotationFactor);
twist = rot(vertexY * twistAmount + uTwistPhase);
```

**Why this works:** A bass hit at t=10s and t=600s both add the same `delta * boost` increment — the visual effect is identical regardless of session duration.

## Direct Audio-Reactive Offsets (Time-Independent)

For instant "punch" response to audio, add a **direct offset** uniform that maps audio energy to a visual parameter without any time multiplication:

```typescript
// Direct offset: audio -> visual, no time involved
uniforms.uTwistReact.value = smoothedMids * 3.0 + smoothedBass * 1.5;
```

```glsl
// Added directly to rotation angles
q.xz *= rot(q.y * uTwist + uTwistPhase + uTwistReact);
```

This provides immediate, bounded reactivity that stays constant over time.

## Safe vs Unsafe Audio Modulation Targets

| Target | Safe Pattern | Unsafe Pattern |
|--------|-------------|----------------|
| Rotation/twist angles | Phase accumulator (`+= delta * rate`) | `elapsedTime * audioSpeed` |
| Object size/radius | Additive offset (`base + audio * scale`) | — |
| Thickness/detail | Additive offset (`base + audio * scale`) | — |
| Particle spawn rate | Direct threshold check | — |
| Color shift | Direct mapping | — |
| Animation speed | Drives phase accumulation rate | Multiplied by growing time |

**Rule of thumb:** If a parameter is multiplied by something that grows monotonically (time, frame count), it must NOT be audio-modulated directly. Instead, audio should modulate the *rate of growth* via a phase accumulator.

## PolySphere as Reference Implementation

`PolySphere.tsx` naturally avoids this bug because:
- `time += 0.01 * speed` — speed is user-controlled, not audio-modulated
- Audio only affects additive properties: radius pulse, face detach offset, particle spawn
- None of these are multiplied by the growing `time` variable

## Smoothing Best Practices

```typescript
// Exponential smoothing (EMA) — good defaults
smoothedBass += (bass - smoothedBass) * 0.15; // fast response
smoothedMids += (mids - smoothedMids) * 0.12; // medium
smoothedHighs += (highs - smoothedHighs) * 0.10; // slower (less jitter)
```

- Higher alpha = faster response, more jitter
- Lower alpha = smoother, more latency
- `analyser.smoothingTimeConstant = 0.8` provides additional FFT-level smoothing

## Checklist for New Visualizers

1. **Never multiply audio-modulated values by elapsed time** — use phase accumulators
2. **Add direct audio offsets** for instant reactivity (twist, scale, glow)
3. **Keep offsets bounded** — `smoothedValue` is already 0–1 range (with sensitivity=1)
4. **Test at 5+ minutes** — compare visual intensity to the first 10 seconds
5. **Sensitivity slider** scales raw band energy before smoothing — works naturally with both patterns
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## [0.19.0] - 2026-06-15

### Added
- Holo Blinds visualizer – raymarched twisting gyroid confined to a squashed sphere core with audio-reactive brightness, twist speed, and detail (Three.js/WebGL).
- Inside Quantum visualizer – full-screen KIFS fractal with volumetric raymarching, asymmetric folding, domain warping, and accumulated glow (Three.js/WebGL). Audio-reactive warp amplitude, rotation speed, and ray thickness.

### Changed
- Dependency bumps

### Acknowledgments
- Shoutout to [@sabosugi](https://x.com/sabosugi) for the nice visuals.

## [0.18.0] - 2026-05-23

### Added
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:25-alpine AS builder
FROM node:26-alpine AS builder
WORKDIR /app

COPY package*.json ./
Expand All @@ -7,7 +7,7 @@ RUN npm ci
COPY . .
RUN npm run build

FROM nginx:1.29-alpine
FROM nginx:1.31.1-alpine

COPY nginx/default.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/dist /usr/share/nginx/html
Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ http://localhost:8080
| `npm run preview` | Preview production build locally |
| `npm run clean` | Remove build artifacts |
| `npm run lint` | Check TypeScript for errors |
| `npm run test` | Run the Playwright smoke tests |
| `npm run test:install` | Download the Playwright Chromium browser |

---

Expand Down Expand Up @@ -199,6 +201,18 @@ npm run build
npm run preview # Test production build locally
```

### Run Smoke Tests
```bash
npm run test:install
npm run test
```

On Linux, Playwright may also need system browser libraries:

```bash
npx playwright install-deps chromium
```

### Environment Setup

The app requires **microphone or display-capture permissions** to function properly. When you first load VoltViz, you'll be prompted to grant these permissions.
Expand Down Expand Up @@ -258,6 +272,12 @@ Contributions are welcome! Whether you want to add new visualizations, improve p

---

## 🎨 Credits

Special shoutout to [@sabosugi](https://x.com/sabosugi) for the nice visuals.

---

## 💡 Tips

- **Best Experience**: Use with headphones and a full-screen window
Expand Down
Loading