Skip to content

Add deterministic effect timing, bursts, envelopes, and coherent variation - #6

Merged
Artsen merged 2 commits into
mainfrom
feat/temporal-effect-modulation
Jul 28, 2026
Merged

Add deterministic effect timing, bursts, envelopes, and coherent variation#6
Artsen merged 2 commits into
mainfrom
feat/temporal-effect-modulation

Conversation

@Artsen

@Artsen Artsen commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • add strict Recipe v2 timing while retaining the exact Recipe v1 path
  • compile deterministic continuous, range, sporadic, and manual-event schedules
  • add bounded duration/cooldown controls, attack/release envelopes, effective intensity, and effect-specific parameter scaling
  • add named per-frame, per-event, and random-access smooth variation channels
  • make horizontal-glitch layouts event-coherent while offsets can drift smoothly
  • add bounded schedule inspection and Recipe v2 timestamp-preview metadata
  • add registry-driven transitional timing controls, schedule summaries, and active-preview feedback
  • bump the application to 0.3.0

Compatibility

  • Recipe v1 remains exact: its JSON contract, continuous enabled-effect behavior, v1 frame-index RNG namespace, image workflow, and persisted records are unchanged
  • golden regression hashes lock representative Recipe v1 frames at indexes 0, 1, and 17
  • manifest schema remains v2
  • storage schema remains v2
  • video-job contract remains v2
  • existing manifest-v2 files containing Recipe v1 load without migration or rewriting
  • image APIs and the image UI continue accepting/building Recipe v1 only
  • video preview and jobs accept Recipe v1 or Recipe v2
  • existing telemetry, cancellation, restart recovery, queue behavior, media streaming, output profile, and audio preservation/removal remain operational

Temporal contract

  • timing modes: continuous, range, sporadic, and events
  • starts use floor(seconds × rational frame rate) and are inclusive; ends use ceil(seconds × rational frame rate) and are exclusive
  • all windows are source-clipped; empty/out-of-source job windows are rejected before persistence
  • manual events are sorted and overlaps after frame conversion are rejected
  • partial sporadic events at the end of a window are omitted
  • cooldown begins after the preceding event ends; an effect's events never overlap
  • schedules are isolated by root seed, stable effect ID, and a Recipe v2 schedule namespace
  • compiled schedules are limited to 10,000 events per effect; inspection returns at most 200 and reports truncation
  • envelope curves: linear, easeIn, easeOut, easeInOut, and sharp
  • effective intensity is configured intensity multiplied by envelope intensity; zero is byte-identical to the input frame
  • named variation channels declare supported/default perFrame, perEvent, and smooth behavior in the registry
  • smooth variation uses deterministic smoothstep interpolation between derived anchors, not mutable random-walk state
  • natural defaults are registry-owned and canonicalized before persistence

Engine architecture

  • MediaTimelineContext, immutable CompiledRecipe, CompiledEffectPlan, EffectEvent, and activation snapshots separate scheduling from rendering
  • recipes compile once per job/request and activation uses binary search over bounded intervals
  • a typed execution context supplies frame/event identity, effective intensity, and isolated named random channels
  • parameter scaling is centralized for all eight existing effects; pixelation also uses fractional blending during ramps
  • horizontal glitch derives layout separately from offset drift so bands can remain recognizable within an event
  • Recipe v1 still calls the original operations with the original RNG and is not routed through the temporal compiler
  • preview, inspection, full processing, and restart recompilation share the same deterministic compiler
  • compiled plans and runtime RNG state are never persisted

UI behavior

  • image mode still builds Recipe v1; video mode builds Recipe v2
  • semantic Advanced timing disclosures appear only for enabled video effects
  • controls cover continuous/range/sporadic timing, start/end, frequency, duration, cooldown, attack/release, curve, intensity, and supported variation modes
  • defaults are loaded from GET /api/effects rather than duplicated in frontend recipe builders
  • schedule requests are debounced, abort obsolete work, and ignore stale revisions
  • the schedule summary reports event counts/times; preview feedback reports active effect IDs
  • the previous valid still remains visible while preview updates
  • controls remain keyboard-operable and responsive in the transitional Flask/JavaScript interface

Validation

  • Python: 263 tests passed
  • statement coverage: 3,555 / 3,655 (97.26%)
  • branch coverage: 720 / 784 (91.84%)
  • combined coverage: 96.31%
  • Ruff format and lint: passed
  • strict Mypy: passed for 36 source files
  • repository consistency and git diff --check: passed
  • Playwright Chromium: 7 workflows passed
  • Axe: zero serious or critical findings in covered image and temporal-video workflows
  • real FFmpeg/FFprobe integration: passed with Recipe v2 processing, H.264/yuv420p output, AAC preservation, telemetry milestones, persisted timing data, and paired Recipe v1 audio removal
  • Recipe v1 golden regressions: passed at frames 0, 1, and 17
  • deterministic schedules: passed across seeds, effect-ID isolation, unrelated parameter changes, reordering-safe namespaces, clipping, event bounds, and serialized restart recompilation
  • envelopes/scaling: explicit curve, attack/hold/release, zero/full intensity, and all-effect typed scaling tests passed
  • coherent variation: per-frame, per-event, smooth random access, held horizontal layouts, later-event layout changes, and smooth offsets passed
  • preview/full processing: exact pre-encode Recipe v2 frame-array equivalence passed
  • controlled browser/manual-equivalent checks cover quiet/active schedule states, natural horizontal-glitch burst defaults, responsive controls, telemetry, cancellation, and image regressions
  • performance: eight continuous effects compiled in 0.32 ms; eight sporadic effects/978 events in 11.81 ms; randomized activation lookup averaged 1.91 µs per effect; a one-hour 60 fps/5,810-event schedule compiled in 60.33 ms on the development machine
  • GitHub Actions: quality and browser passed

Known limitations

  • the current UI is not a full editable timeline and has no draggable keyframes
  • manual event editing is API-only in this PR
  • preview remains a still frame rather than a preview clip
  • smooth variation is deterministic value interpolation, not optical flow
  • temporal modulation does not use previous frames, feedback buffers, trails, or blending state
  • codec datamoshing and motion-vector manipulation remain deferred processing domains
  • the interface remains transitional Flask/JavaScript; React and Vite remain deferred
  • one video worker remains the default
  • manifest writes remain single-process
  • no authentication exists and the application remains loopback-only

@Artsen
Artsen marked this pull request as ready for review July 28, 2026 04:51
@Artsen
Artsen merged commit ea1f9e8 into main Jul 28, 2026
2 checks passed
@Artsen
Artsen deleted the feat/temporal-effect-modulation branch July 28, 2026 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant