Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,17 @@ timing summaries survive restart; rolling speed and ETA restart as unknown.
FFmpeg stdout carries structured program progress while a separate reader drains
bounded stderr. The polling UI owns one timer and one request, rejects obsolete
job revisions, and stops on terminal or reset states.

Recipe v2 adds a separate temporal path without changing Recipe v1 execution.
The strict contract is canonicalized with registry-owned video defaults, then a
`MediaTimelineContext` and immutable `CompiledRecipe` are built once. Each
effect plan stores bounded sorted intervals and supports binary-search
activation. Preview, schedule inspection, full processing, and restart
recompilation use this same compiler.

Scheduling owns activation and envelope intensity. A typed execution context
owns named deterministic random channels. Operations own visual transformation
and centralized parameter scaling. This separation keeps routes and the video
loop free of effect-specific timing rules. Smooth variation derives random
anchors on demand and is independent of processing order; no mutable global RNG
or previous-frame buffer exists.
12 changes: 9 additions & 3 deletions docs/product-direction.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,15 @@ delivery-optimization and packaging tool.

The transitional interface explains long video work with concise, progressively
disclosed telemetry. It truthfully describes applying the enabled stack to each
frame rather than pretending effects run as separate passes. Advanced temporal
feedback, optical flow, frame reordering, codec datamosh, signal modeling, and
new effect algorithms remain intentionally deferred.
frame rather than pretending effects run as separate passes. Recipe v2 now adds
deterministic ranges and bursts, attack/release envelopes, natural parameter
scaling, and coherent named variation to those existing effects. The compiled
schedule API is a stable base for a future React timeline.

Stateful temporal feedback, optical flow, frame reordering, codec datamosh,
signal modeling, new effect algorithms, draggable keyframes, and preview clips
remain intentionally deferred. Temporal modulation changes independent decoded
frames and does not use prior-frame state.

A future orchestration dashboard may discover and check GlitchCraft, ColorCraft,
and Web Video Optimizer through related contracts. It is not implemented here
Expand Down
86 changes: 86 additions & 0 deletions docs/recipes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Effect recipes

GlitchCraft 0.3.0 supports recipe schema versions 1 and 2. Recipe documents are
independently versioned inside manifest-v2 records, so adding Recipe v2 does not
change the manifest, storage, or video-job schemas.

## Recipe v1: exact legacy behavior

Recipe v1 remains the default for still images and is supported indefinitely for
persisted sources, outputs, and jobs. Its enabled effects run on every frame.
Each stochastic effect receives the existing
`glitchcraft:v1:<seed>:<effect-id>:<frame-index>` random namespace. Temporal
fields are rejected.

The v1 engine was not routed through the temporal compiler. Golden frame hashes
lock representative v1 output at multiple frame indexes, which protects image
exports, interrupted video jobs, and deterministic fixtures from reinterpretation.

## Recipe v2: temporal modulation

Recipe v2 retains the root seed, ordered instances, stable unique IDs, enabled
state, effect types, and strict parameter models. Each instance also has:

- `intensity`, from 0 through 1;
- `timing`, using `continuous`, `range`, `sporadic`, or `events`;
- `envelope`, with attack, release, and a curve;
- `variation`, keyed by registry-declared random channel.

Unknown fields, effect IDs, channels, modes, parameters, and mode-specific timing
fields are rejected. The canonical JSON uses camel-case aliases and can be
persisted without runtime state.

```json
{
"schemaVersion": 2,
"seed": 184729,
"effects": [
{
"id": "horizontal-glitch",
"type": "horizontal_glitch",
"enabled": true,
"intensity": 1,
"parameters": {"count": 6, "shift": 80},
"timing": {
"mode": "sporadic",
"startSeconds": 0,
"endSeconds": null,
"frequencyPerMinute": 8,
"minimumDurationFrames": 6,
"maximumDurationFrames": 18,
"minimumCooldownFrames": 24,
"maximumCooldownFrames": 90
},
"envelope": {
"attackFrames": 1,
"releaseFrames": 5,
"curve": "easeOut"
},
"variation": {
"layout": {"mode": "perEvent"},
"offset": {"mode": "smooth", "periodFrames": 6}
}
}
]
}
```

Natural video defaults belong to the Python effect registry and are returned by
`GET /api/effects`. The transitional frontend consumes that metadata rather than
maintaining another copy. An explicit instance value overrides its default.
Recipe v1 never receives these defaults.

## Still-image rule

Image requests continue accepting Recipe v1 only. This strict rule prevents a
sporadic video schedule from making a still effect unexpectedly disappear.
Image preview and export behavior therefore remain byte-compatible. Recipe v2
is accepted by the video preview, schedule, and job APIs.

## Persistence and restart

Manifest-v2 image records contain Recipe v1. Manifest-v2 video jobs and outputs
may contain Recipe v1 or Recipe v2. Existing files load without migration or
rewriting. Recipe v2 persists only its portable contract; compiled intervals and
random-generator state are never stored. Restart recovery recompiles the same
schedule from the recipe, source timeline, effect ID, and isolated namespaces.
16 changes: 15 additions & 1 deletion docs/service-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,24 @@ state. Readiness reports queue capacity, worker concurrency, manifest migration
or reconciliation state, and both required video tools. Saved recipe management,
WVO handoff, authentication, and a visible library interface are deferred.

Application version 0.2.1 advertises video-job contract v2 and the
Application version 0.3.0 advertises video-job contract v2 and the
`video-render-telemetry` capability. `/metadata` publishes supported phase
codes, the client-polling model, a four-Hz runtime update target, FFmpeg
program-progress support, and redacted queue totals. The capability is available
only when persistent video processing and its manager are available. Discovery
never exposes sample buffers, thread names, executable paths, or raw process
output.

The current recipe schema is 2 and supported recipe schemas are `[1, 2]`.
Manifest and storage schemas remain 2. `GET /api/effects` returns registry-owned
temporal defaults, envelope support, intensity scaling, and named variation
channels. `POST /api/video-sources/{sourceId}/effect-schedule` accepts Recipe v2
and returns a bounded redacted schedule. Preview responses expose only safe
actual-frame and active-effect headers.

Temporal capability slugs are `temporal-effect-modulation`,
`deterministic-effect-schedules`, `effect-envelopes`, and
`coherent-effect-variation`. Metadata lists continuous, range, sporadic, and
events timing; all five envelope curves; and per-frame, per-event, and smooth
variation. It does not claim feedback, optical flow, datamoshing, keyframes, or
timeline editing.
11 changes: 9 additions & 2 deletions docs/storage.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Persistent media storage

GlitchCraft 0.2.1 uses a configurable managed data root:
GlitchCraft 0.3.0 uses a configurable managed data root:

```text
data/
Expand Down Expand Up @@ -58,8 +58,15 @@ diagnostics whose values and granularity may vary by platform.

Video completion installs the final MP4 and marks its job completed in one
manifest mutation. Intermediate job files remain temporary and are never served.
Manifest schema remains v2 in application 0.2.1. Telemetry contract v2 adds only
Manifest schema remains v2 in application 0.3.0. Telemetry contract v2 adds only
backward-compatible job defaults: a phase, at most 32 durable milestones, a
terminal timing summary, and coarse last-known frame counters. High-frequency
FPS, ETA, encoded-time, queue-position, and stale calculations are runtime-only
and never cause per-frame manifest writes.

Manifest-v2 recipe fields are independently discriminated and may contain
Recipe v1 or Recipe v2. Existing Recipe v1 JSON validates directly and is not
migrated or rewritten. Recipe v2 jobs and outputs retain timing, envelope,
intensity, and variation configuration, but not compiled events or runtime RNG
state. Schedule compilation after restart is deterministic and does not add
per-frame persistence.
131 changes: 131 additions & 0 deletions docs/temporal-effects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Deterministic temporal effects

Temporal modulation controls when and how strongly existing decoded-frame
effects run. It does not add new effect types, retain previous frames, perform
optical flow, or manipulate codec structures.

## Timeline and frame conversion

`MediaTimelineContext` contains the rational source frame rate, source duration,
total frames, and media type. A starting time is converted with
`floor(seconds × frame rate)` and is inclusive. An ending time is converted with
`ceil(seconds × frame rate)` and is exclusive. Both are clipped to the source.
The actual preview timestamp is the selected frame index divided by the same
rational rate.

Video preview seeks to that frame index for Recipe v2. Schedule inspection and
full processing use the same compiler, so a direct preview of frame 250 has the
same activation, envelope, and variation as frame 250 in sequential rendering.

## Timing modes

- `continuous` produces one interval over the source, optionally bounded by
start and end times.
- `range` requires one finite start and later finite end.
- `sporadic` precompiles deterministic, non-overlapping bursts. Frequency,
duration, and cooldown ranges are bounded. Cooldown begins after the preceding
event. A partial final event is omitted.
- `events` accepts a nonempty manually placed list. Events are sorted, start is
inclusive, end is exclusive, and overlaps after frame conversion are rejected.

Each effect is limited to 10,000 compiled events. The inspection response returns
at most 200 events per effect and reports truncation. Invalid or out-of-source
windows are rejected before a persistent job is created.

Sporadic random values use a schedule namespace derived from the root seed and
stable effect-instance ID. Parameter edits, unrelated effects, and effect order
do not perturb another instance's schedule. Activation is not rolled
independently on every frame.

## Compiled plan

`CompiledRecipe` owns immutable `CompiledEffectPlan` values. Each event records
the effect ID, event index, inclusive start, exclusive end, duration, and stable
identity. Start-frame indexes support bounded binary-search activation lookup.
Compilation occurs once per video job or request; schedules are not regenerated
inside the frame loop.

An activation snapshot provides event position, duration, configured intensity,
envelope intensity, effective intensity, and event identity. Inactive effects
are skipped before their operation runs.

## Envelopes and intensity

Supported curves are `linear`, `easeIn`, `easeOut`, `easeInOut`, and `sharp`.
Attack and release lengths are nonnegative and may not exceed the shortest
possible event when combined.

An instant attack begins at full strength. For an attack of N frames, the first
frame evaluates the curve at `1/N` and frame N−1 reaches one. A release of N
frames begins N frames before the exclusive end; the final interval frame
evaluates to zero. Frames outside the event are exactly zero.

`effective intensity = configured intensity × envelope value`

At zero, the operation is skipped and the frame is byte-identical. At one,
configured parameters are fully applied.

## Natural parameter scaling

Scaling is centralized in the effect execution layer:

- noise amount and strength move toward zero;
- pixel size moves toward one, with fractional blending during ramps;
- horizontal band count and displacement move toward zero;
- frame X/Y displacement moves toward zero;
- color-channel offsets move toward zero;
- scan-line darkness moves toward one;
- static density moves toward zero;
- flicker bounds move toward one.

All scaled values are rebuilt through typed, validated parameter models. Array
math uses bounded float intermediates and returns RGB `uint8` without overflow.

## Named variation channels

The registry declares each supported channel, allowed modes, default mode, and
smooth period:

- noise and static: `detail`;
- horizontal glitch: `layout` and `offset`;
- frame shift: `offset`;
- color bleed: `channels`;
- flicker: `level`;
- pixelation and scan lines: no random channel.

`perFrame` derives a value from the absolute frame. `perEvent` derives it from
the event index and holds it. `smooth` derives neighboring deterministic anchors
and interpolates with smoothstep. Smooth values require no mutable random walk,
so random-access preview, sequential rendering, restart, and evaluation order
agree.

Horizontal glitch uses an event-held layout and separately sampled offsets.
With its natural defaults, recognizable bands persist while offsets drift,
envelope-scaled displacement settles, quiet frames follow, and a later event can
select a different layout. Static retains per-frame detail variation.

## APIs and current UI

`GET /api/effects` returns parameter metadata, temporal defaults, intensity
support, and channel contracts. `POST
/api/video-sources/<source-id>/effect-schedule` returns a bounded compiled
schedule. Recipe v2 preview responses include safe headers for the actual frame,
actual timestamp, active effect IDs, and effective intensities.

Video mode builds Recipe v2. Semantic Advanced timing disclosures expose
continuous, range, and sporadic controls, duration/cooldown, envelope, intensity,
and variation. Schedule requests are debounced, abort obsolete work, and ignore
stale responses. The previous valid still preview stays visible while updating.
Image mode stays Recipe v1.

This remains a transitional Flask/JavaScript interface. It has no draggable
timeline or complete manual-event editor, and preview remains a still frame.
Future React timeline work can consume the schedule endpoint without changing
the recipe or compiler.

## Domain boundaries

Temporal modulation evaluates independent decoded frames. It does not use
previous-frame feedback, frame echo, motion trails, optical flow, or stateful
blending. Codec datamoshing changes compressed-frame or motion-vector structures
and remains a separate future processing domain.
25 changes: 25 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,28 @@ finalizing, verifying, saving, completed, cancel-pending, and canceled states.
It checks concise formatting, keyboard-accessible details, ordered effects,
previous-result retention, responsive containment, and Axe results without
requiring a long render.

Recipe v1 golden regressions hash representative frames at indexes 0, 1, and 17
to prove the legacy RNG namespace and pixels remain exact. Temporal tests cover
strict Recipe v2 parsing, rational conversion, every timing mode, event sorting
and overlap rejection, clipping, event limits, schedule seed/ID isolation,
restart recompilation, every envelope curve, intensity scaling for all eight
effects, and random-access per-frame/per-event/smooth variation.

Pre-encode arrays prove Recipe v2 preview and sequential processing equivalence.
The real FFmpeg/FFprobe test runs a Recipe v2 job through processing, H.264
finalization, AAC preservation, telemetry, and persisted timing metadata; its
second Recipe v1 job verifies audio removal and legacy video compatibility.
Playwright verifies video mode sends Recipe v2 while image mode sends Recipe v1,
natural horizontal-glitch defaults, timing disclosure keyboard behavior,
schedule summaries, active-preview feedback, responsive controls, existing job
telemetry/cancellation, and no serious or critical Axe findings.

On the development machine, 200-iteration temporal benchmarks measured an
eight-effect continuous compile at 0.32 ms and an eight-effect sporadic compile
at 11.81 ms for 978 events. Ten thousand randomized timestamp probes across
those eight sporadic plans averaged 1.91 microseconds per effect lookup. A
one-hour 60 fps synthetic schedule containing 5,810 events compiled in 60.33 ms.
These are diagnostic measurements, not platform guarantees; bounded event
counts, one compile per job, binary-search lookup, and inactive-operation
skipping are the enforced properties.
13 changes: 12 additions & 1 deletion docs/video-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ and job are committed in one manifest mutation.

## Render telemetry contract v2

Application 0.2.1 keeps the persistent states above and adds more precise phase
Application 0.2.1 introduced the more precise phase
codes: `waiting`, `preparing_source`, `applying_effects`,
`finalizing_output`, `verifying_output`, `saving_output`, `completed`,
`failed`, and `canceled`.
Expand All @@ -46,3 +46,14 @@ defaults. Only bounded phase milestones, coarse frame progress, and a terminal
timing summary are durable. FFmpeg finalization uses program-progress output and
separately drains bounded diagnostics. Unknown keys and malformed optional
values are ignored; neither diagnostics nor reader-thread details enter the API.

Jobs accept and persist Recipe v1 or Recipe v2 while the video-job contract
remains v2. Recipe v2 is compiled once after source metadata is available;
inactive effects are skipped without changing frame telemetry or cancellation.
Restarted jobs rebuild the same plan from their persisted recipe and source
timeline. Compiled intervals and random state are not manifest data.

The concise telemetry phrase continues counting enabled instances, not active
effects on one scheduled frame. Event-level changes do not create milestones or
manifest writes. Individual job and output metadata expose `recipeVersion` so
clients can distinguish legacy-continuous work from temporal work.
14 changes: 14 additions & 0 deletions docs/video-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,17 @@ to constant frame rate. Only the first audio stream is preserved. Subtitles,
chapters, attachments, multiple audio tracks, and hardware encoding are not
preserved or implemented. Detailed delivery optimization remains the
responsibility of Web Video Optimizer.

Video mode now builds Recipe v2 from `/api/effects` registry metadata. Advanced
timing disclosures appear only for enabled video effects and expose continuous,
range, and sporadic modes, bounds, frequency, duration, cooldown, attack,
release, curve, maximum intensity, and supported variation choices. Image mode
continues to build Recipe v1.

Schedule and preview requests are debounced independently, abort obsolete
requests, and reject stale revisions. The schedule summary shows bounded event
locations. Preview headers report the actual rationally selected frame and
active effects. Moving the timestamp never changes the root seed or schedule,
and the previous valid still remains visible while its replacement loads.
Manual events are supported by the API contract but do not yet have a full
editor.
Loading
Loading