diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..4abee26 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,52 @@ + + +# Changelog + +All notable changes to the Podscript specification are recorded here. The spec follows +[Semantic Versioning](https://semver.org/) (`MAJOR.MINOR.PATCH`, SPEC §20): a change that +alters the meaning of an existing valid script or any golden conformance fixture is +**major**; additive, backward-compatible features are **minor**; clarifications are +**patch**. Scripts declare the lowest version whose features they use (§1.2). + +## 0.2.0 — 2026-07-03 + +### Added +- **Prosody model (SPEC §5.5).** A `prosody` control for speaking delivery — the continuous + attributes `rate` (tempo multiplier), `pitch` (semitone shift), and `volume` (vocal-intensity + multiplier) — as a per-voice baseline object (§2.3) and a per-line `cue` clause (§5), merged + **per attribute** onto the IR clip (§11.1) and omitted at natural defaults. The section maps + the remaining prosodic dimensions onto existing constructs (`*emphasis*`, breaks, sentence + punctuation, `(direction)`). +- New `SEMITONES` token (§1.5); AST `Voice.prosody`, `Prosody` interface, and `prosody` clause + variant (§10); IR-clip `prosody` (§11). +- Prosody is **advisory** synthesis guidance — a deliberate exception to fail-closed (§15.4): + core vocabulary, but engines drop attributes they cannot honour, degrading expressiveness not + mix correctness. It joins the synthesis cache key (§14.3) and does **not** affect + timing/resolution. +- Conformance fixtures `conformance/parse/prosody` and `conformance/resolve/prosody`; runnable + example `examples/prosody.podscript`. + +### Changed +- Adopted three-part semantic-version stamps (`MAJOR.MINOR.PATCH`; `PATCH` optional, defaults + to `0`, §1.2). Normalized every bundled example and conformance-fixture stamp from `0.1` to + `0.2.0` so the repository uses one consistent current version — a one-time pre-1.0 housekeeping + step. Two-part `0.1` stamps remain valid input (`0.1` ≡ `0.1.0`). + +### Compatibility +- The prosody feature is additive and backward-compatible at the language level: `prosody` is + omitted when absent, and a `0.1` script parses unchanged under a 0.2.0 engine. The version + *stamps* in the bundled examples and golden fixtures were re-stamped to `0.2.0` as part of the + semver normalization above — a housekeeping change to the stamp only, not a semantic change to + any script's meaning. + +## 0.1.0 — 2026-06-24 + +### Added +- Initial public draft: lexical grammar; `meta` / `voices` / `assets` / `pronounce` headers; + speech lines with typed inline content (emphasis, breaks, pronunciation); clauses (anchors, + `gain`, `fade`, `crossfade`, `duck-under`, `at`, loop); the relational timing model and speech + spine; the resolved IR schema; the gain model; determinism and the render manifest; + extensibility (namespaced extensions, fail-closed); security; and transcript export. diff --git a/README.md b/README.md index 816b10a..17adc22 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ music beds, sound effects, fades, and ducking — that compiles deterministicall finished audio file. ```podscript -podscript: "0.1" +podscript: "0.2.0" voices: sam: { voice: Adam, preset: host } @@ -69,7 +69,7 @@ and any renderer. Source → IR → audio. ## The specification -[`docs/SPEC.md`](docs/SPEC.md) is the normative v0.1 specification: lexical grammar, +[`docs/SPEC.md`](docs/SPEC.md) is the normative v0.2.0 specification: lexical grammar, speakers and voices, cues and directions, the gain / duck / fade model, the relational timing model, the IR schema, determinism, extensibility, security, and transcript export. This README is informative; where they differ, the spec wins. Domain terms (bed, duck, @@ -91,7 +91,9 @@ provider-dependent and reproducible only when pinned via a render manifest. ## Examples [`examples/`](examples/) holds runnable sample scripts, including -[`cold_open.podscript`](examples/cold_open.podscript). +[`cold_open.podscript`](examples/cold_open.podscript) and +[`prosody.podscript`](examples/prosody.podscript) (controlling delivery — rate, pitch, +volume, emphasis). ## Implementations diff --git a/conformance/README.md b/conformance/README.md index fcd08de..c264d92 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -38,6 +38,8 @@ renderer) and MUST pass every fixture for each class it claims. | `resolve` | `full_episode` | the above plus: continuous cross-scene timeline, bed placement at the playhead, bed lifetime across scenes, two instances of one asset, `duck-under speech` (declarative), and a cross-scene `crossfade` paired with a forward-anchored (`at signoff.end`) fade-out | | `parse` | `pronounce` | `pronounce:` header lexicon (quoted multi-word term), inline `{say "X" as "Y"}` pronunciation node alongside emphasis/break (§2.5, §3.1) | | `resolve` | `pronounce` | `pronounce` map copied to the IR root; speech `content` (incl. the inline `pronounce` node) preserved verbatim with original spelling; timing unaffected | +| `parse` | `prosody` | voice-level `prosody: { rate, pitch }` baseline object and per-line `prosody rate/pitch/volume` clauses in the speech `cue` (§2.3, §5.5); pitch parsed from semitones (`-1st` → `-1`); a voice without `prosody` omits it | +| `resolve` | `prosody` | per-attribute effective-prosody merge onto the clip (line clause overrides voice baseline attribute-by-attribute; attributes at their natural default and the empty object omitted); prosody is synthesis metadata and does not perturb `start`/`dur`/`end` | The semantic questions the multi-scene episode raised — multi-scene timing, bed lifetime/placement, cross-scene crossfade, duck envelope, and how to add automation diff --git a/conformance/parse/cold_open/expected.ast.json b/conformance/parse/cold_open/expected.ast.json index 6a91f48..592fa5e 100644 --- a/conformance/parse/cold_open/expected.ast.json +++ b/conformance/parse/cold_open/expected.ast.json @@ -1,5 +1,5 @@ { - "podscript": "0.1", + "podscript": "0.2.0", "meta": { "title": "Episode 12 — The Thing", "lufs": -16, diff --git a/conformance/parse/cold_open/input.podscript b/conformance/parse/cold_open/input.podscript index 78f1219..45b436f 100644 --- a/conformance/parse/cold_open/input.podscript +++ b/conformance/parse/cold_open/input.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" meta: title: "Episode 12 — The Thing" diff --git a/conformance/parse/pronounce/expected.ast.json b/conformance/parse/pronounce/expected.ast.json index 5f2c7d1..176570c 100644 --- a/conformance/parse/pronounce/expected.ast.json +++ b/conformance/parse/pronounce/expected.ast.json @@ -1,5 +1,5 @@ { - "podscript": "0.1", + "podscript": "0.2.0", "meta": {}, "requires": [], "voices": { diff --git a/conformance/parse/pronounce/input.podscript b/conformance/parse/pronounce/input.podscript index 8c2cd1a..32b9e77 100644 --- a/conformance/parse/pronounce/input.podscript +++ b/conformance/parse/pronounce/input.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" voices: sam: { voice: Adam, preset: host } diff --git a/conformance/parse/prosody/expected.ast.json b/conformance/parse/prosody/expected.ast.json new file mode 100644 index 0000000..6888c33 --- /dev/null +++ b/conformance/parse/prosody/expected.ast.json @@ -0,0 +1,90 @@ +{ + "podscript": "0.2.0", + "meta": {}, + "requires": [], + "voices": { + "sam": { + "voice": "Adam", + "preset": "host", + "prosody": { + "rate": 1.1, + "pitch": -1 + } + }, + "alex": { + "voice": "Rachel", + "preset": "warm-host" + } + }, + "assets": {}, + "scenes": [ + { + "kind": "scene", + "name": "chat", + "body": [ + { + "kind": "speech", + "speaker": "sam", + "content": [ + { + "kind": "text", + "value": "This is my normal quicker cadence." + } + ] + }, + { + "kind": "speech", + "speaker": "sam", + "cue": [ + { + "kind": "prosody", + "rate": 1.35, + "pitch": 3, + "volume": 1.2 + } + ], + "content": [ + { + "kind": "text", + "value": "Wait, it actually " + }, + { + "kind": "emphasis", + "value": "worked" + }, + { + "kind": "text", + "value": "!" + } + ] + }, + { + "kind": "speech", + "speaker": "alex", + "cue": [ + { + "kind": "prosody", + "rate": 0.9 + } + ], + "content": [ + { + "kind": "text", + "value": "Slow and steady over here." + } + ] + }, + { + "kind": "speech", + "speaker": "alex", + "content": [ + { + "kind": "text", + "value": "And back to normal." + } + ] + } + ] + } + ] +} diff --git a/conformance/parse/prosody/input.podscript b/conformance/parse/prosody/input.podscript new file mode 100644 index 0000000..8a3618e --- /dev/null +++ b/conformance/parse/prosody/input.podscript @@ -0,0 +1,11 @@ +podscript: "0.2.0" + +voices: + sam: { voice: Adam, preset: host, prosody: { rate: 1.1, pitch: -1st } } + alex: { voice: Rachel, preset: warm-host } + +scene chat: + sam: This is my normal quicker cadence. + sam [prosody rate 1.35 pitch +3st volume 1.2]: Wait, it actually *worked*! + alex [prosody rate 0.9]: Slow and steady over here. + alex: And back to normal. diff --git a/conformance/resolve/cold_open/expected.ir.json b/conformance/resolve/cold_open/expected.ir.json index 677c155..e31d7aa 100644 --- a/conformance/resolve/cold_open/expected.ir.json +++ b/conformance/resolve/cold_open/expected.ir.json @@ -1,5 +1,5 @@ { - "podscript": "0.1", + "podscript": "0.2.0", "meta": { "title": "Episode 12 — The Thing", "lufs": -16, diff --git a/conformance/resolve/cold_open/input.podscript b/conformance/resolve/cold_open/input.podscript index 78f1219..45b436f 100644 --- a/conformance/resolve/cold_open/input.podscript +++ b/conformance/resolve/cold_open/input.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" meta: title: "Episode 12 — The Thing" diff --git a/conformance/resolve/full_episode/expected.ir.json b/conformance/resolve/full_episode/expected.ir.json index 38b7cfb..75913f3 100644 --- a/conformance/resolve/full_episode/expected.ir.json +++ b/conformance/resolve/full_episode/expected.ir.json @@ -1,5 +1,5 @@ { - "podscript": "0.1", + "podscript": "0.2.0", "meta": { "title": "Episode 12 — The Thing", "lufs": -16, diff --git a/conformance/resolve/full_episode/input.podscript b/conformance/resolve/full_episode/input.podscript index 56186cc..fd4f6dc 100644 --- a/conformance/resolve/full_episode/input.podscript +++ b/conformance/resolve/full_episode/input.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" # Full multi-scene episode: exercises continuous cross-scene timeline, # bed lifetime across scenes, duck-under-speech, and a cross-scene crossfade diff --git a/conformance/resolve/pronounce/expected.ir.json b/conformance/resolve/pronounce/expected.ir.json index 7cb77d2..497368a 100644 --- a/conformance/resolve/pronounce/expected.ir.json +++ b/conformance/resolve/pronounce/expected.ir.json @@ -1,5 +1,5 @@ { - "podscript": "0.1", + "podscript": "0.2.0", "meta": {}, "pronounce": { "Seibold": "SY-bold", diff --git a/conformance/resolve/pronounce/input.podscript b/conformance/resolve/pronounce/input.podscript index 8c2cd1a..32b9e77 100644 --- a/conformance/resolve/pronounce/input.podscript +++ b/conformance/resolve/pronounce/input.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" voices: sam: { voice: Adam, preset: host } diff --git a/conformance/resolve/prosody/durations.json b/conformance/resolve/prosody/durations.json new file mode 100644 index 0000000..e1fd4e3 --- /dev/null +++ b/conformance/resolve/prosody/durations.json @@ -0,0 +1,10 @@ +{ + "_comment": "Durations supplied to the resolver, in milliseconds, standing in for TTS + ffprobe. Speech keyed by clip id; assets by asset id. Prosody does not affect resolution, so these are the measured clip lengths regardless of any rate/pitch/volume.", + "clips": { + "c1": 3000, + "c2": 1800, + "c3": 2600, + "c4": 1500 + }, + "assets": {} +} diff --git a/conformance/resolve/prosody/expected.ir.json b/conformance/resolve/prosody/expected.ir.json new file mode 100644 index 0000000..0bf8019 --- /dev/null +++ b/conformance/resolve/prosody/expected.ir.json @@ -0,0 +1,88 @@ +{ + "podscript": "0.2.0", + "meta": {}, + "clips": [ + { + "id": "c1", + "track": "speech", + "voice": "sam", + "preset": "host", + "prosody": { + "rate": 1.1, + "pitch": -1 + }, + "start": 0.0, + "dur": 3.0, + "end": 3.0, + "content": [ + { + "kind": "text", + "value": "This is my normal quicker cadence." + } + ] + }, + { + "id": "c2", + "track": "speech", + "voice": "sam", + "preset": "host", + "prosody": { + "rate": 1.35, + "pitch": 3, + "volume": 1.2 + }, + "start": 3.25, + "dur": 1.8, + "end": 5.05, + "content": [ + { + "kind": "text", + "value": "Wait, it actually " + }, + { + "kind": "emphasis", + "value": "worked" + }, + { + "kind": "text", + "value": "!" + } + ] + }, + { + "id": "c3", + "track": "speech", + "voice": "alex", + "preset": "warm-host", + "prosody": { + "rate": 0.9 + }, + "start": 5.3, + "dur": 2.6, + "end": 7.9, + "content": [ + { + "kind": "text", + "value": "Slow and steady over here." + } + ] + }, + { + "id": "c4", + "track": "speech", + "voice": "alex", + "preset": "warm-host", + "start": 8.15, + "dur": 1.5, + "end": 9.65, + "content": [ + { + "kind": "text", + "value": "And back to normal." + } + ] + } + ], + "beds": [], + "oneshots": [] +} diff --git a/conformance/resolve/prosody/input.podscript b/conformance/resolve/prosody/input.podscript new file mode 100644 index 0000000..8a3618e --- /dev/null +++ b/conformance/resolve/prosody/input.podscript @@ -0,0 +1,11 @@ +podscript: "0.2.0" + +voices: + sam: { voice: Adam, preset: host, prosody: { rate: 1.1, pitch: -1st } } + alex: { voice: Rachel, preset: warm-host } + +scene chat: + sam: This is my normal quicker cadence. + sam [prosody rate 1.35 pitch +3st volume 1.2]: Wait, it actually *worked*! + alex [prosody rate 0.9]: Slow and steady over here. + alex: And back to normal. diff --git a/docs/AUTHORING.md b/docs/AUTHORING.md index dc1762d..2cd61e0 100644 --- a/docs/AUTHORING.md +++ b/docs/AUTHORING.md @@ -9,15 +9,16 @@ This is a compact, self-contained guide to writing a valid `.podscript` file. It intended to be pasted into a model's context. It is **informative**; the normative source is [`SPEC.md`](SPEC.md). When something here is unclear or unstated, follow `SPEC.md`. -**Golden rule:** the output must *parse*. The first non-blank, non-comment line is always -`podscript: "0.1"`. Every speaker must be declared in `voices:`; every asset referenced by +**Golden rule:** the output must *parse*. The first non-blank, non-comment line is always the +version stamp — `podscript: "0.2.0"` (the current version). Every speaker must be declared in +`voices:`; every asset referenced by `bed`/`sfx` must be declared in `assets:`. Levels are in dB; times carry a unit (`s`/`ms`). ## Minimal example ```podscript -podscript: "0.1" +podscript: "0.2.0" voices: host: { voice: Adam, preset: host } @@ -43,7 +44,7 @@ A file is a version line, then optional header blocks **in any order**, then one `scene` blocks. Indent block bodies by 2 spaces. ```podscript -podscript: "0.1" +podscript: "0.2.0" # Comments are whole-line only — a line whose first character is '#'. (A '#' after # text is a label, not a comment; see Speech lines.) Blank lines are ignored. @@ -76,8 +77,9 @@ Block-by-block: - `meta:` (optional) — `title`, `description`, `lufs` (loudness target, default −16), `true_peak` (dBTP ceiling, default −1). - `voices:` (required if there's dialogue) — `speaker-id: { fields }`. Fields: `voice` - (required), `provider`, `preset`, `style`, `language`. The speaker-id is what you use on - dialogue lines. + (required), `provider`, `preset`, `style`, `language`, `prosody`. The speaker-id is what + you use on dialogue lines. `prosody: { rate, pitch, volume }` sets the speaker's baseline + delivery (e.g. `prosody: { rate: 1.1, pitch: -1st }`); see *Prosody* below. - `assets:` (required if any `bed`/`sfx` is used) — `id: path [loop|once]` (default `once`). - `pronounce:` (optional) — `term: "respelling"`; quote keys that contain spaces or punctuation (e.g. `"Dee Why"`). @@ -142,6 +144,7 @@ Clauses (each at most once per directive/cue, order-independent): | At | `at 8s` / `at 0:23` / `at signoff.end+3s` | Place at an absolute time or an anchor. | | Loop | `loop` / `once` | Override the asset's default repeat behaviour. | | Anchor | `after ` / `with ` / `before ` | Relative placement (see below). | +| Prosody | `prosody rate 1.15 pitch +2st volume 1.1` | Speech-only: delivery for this line — any of `rate`/`pitch`/`volume` (`1.0`/`+0st` = natural). | Units are required: levels end in `db`, durations in `s` or `ms`, timecodes are `8s`, `500ms`, or `M:SS(.ms)` like `0:23`. @@ -164,9 +167,81 @@ just labels and do **not** reset time. You only place things relationally: - A **bed** with no anchor starts at the current playhead and plays until it's faded out, replaced by the next bed, or its file ends; beds span scenes. +## Prosody + +**Prosody** is *how* a line is spoken — the delivery on top of the words. Every prosody +control is a hint to the voice engine and never touches timing: the mix is unchanged, only +the speech audio differs. Podscript covers the full range of prosody with a mix of the +`prosody` control and a few constructs you've already seen: + +| Prosodic dimension | Write it as | +|---|---| +| **Rate** (tempo) | `prosody rate 1.15` — multiplier, `1.0` = natural | +| **Pitch** (baseline) | `prosody pitch +2st` / `-1st` — semitone shift | +| **Volume** (vocal effort) | `prosody volume 1.2` — multiplier; *not* the mix level (`gain`) | +| **Emphasis** (stress on a word) | `*word*` inside the text | +| **Pauses** (rhythm) | `...` (a beat) or `{break 500ms}` (exact) | +| **Question vs statement** (intonation) | sentence punctuation — `?` `.` `!` | +| **Emotion / attitude** | `(direction)` — e.g. `(excited)`, `(somber)` | + +The three continuous knobs (`rate`, `pitch`, `volume`) travel together in one `prosody` +control. Set a **baseline** on the voice for a character's habitual delivery, and +**override any attribute per line** (unset attributes keep the baseline): + +```podscript +voices: + sam: { voice: Adam, preset: host, prosody: { rate: 1.1, pitch: -1st } } # Sam: brisk, low + +scene chat: + sam: Normal delivery for the setup. + sam [prosody rate 1.35 pitch +3st volume 1.2]: Wait — it actually *worked*! # this line pops + sam [prosody rate 0.9]: ...which, if you think about it, changes everything. # keeps pitch -1st +``` + +`(direction)` is a free-text emotional note; any words work and the engine maps what it can. +A small, consistent vocabulary keeps LLM-authored scripts predictable: `excited`, `measured`, +`urgent`, `warm`, `deadpan`, `skeptical`, `somber`. + +**Within a sentence, some parts different from others?** There is no per-word prosody — real +voice engines don't expose one. Author it by **splitting the thought into consecutive cues**, +each with its own prosody; the speech spine chains them tightly: + +```podscript +sam (measured): So here's the thing. +sam [prosody rate 1.3 pitch +2st] (excited): It actually worked, first try, no changes! +``` + +**Portability note:** engines honour what they can and drop the rest — `rate` works almost +everywhere; `pitch`/`volume` vary. On engines that don't take fine markup (e.g. ElevenLabs, +which reads punctuation and bracketed audio tags rather than SSML), emphasis, pauses, and +punctuation still land, because they map onto cues the engine already understands. + +Putting it together — the repo's [`examples/prosody.podscript`](../examples/prosody.podscript): + + +```podscript +podscript: "0.2.0" + +# prosody.podscript — controlling delivery +# +# Prosody is *how* a line is spoken. Sam has a brisk, slightly low baseline; +# individual lines override rate / pitch / volume. Within-sentence variation is +# authored as consecutive cues, each with its own prosody. + +voices: + sam: { voice: Adam, preset: host, prosody: { rate: 1.1, pitch: -1st } } + alex: { voice: Rachel, preset: warm-host } + +scene demo: + sam: Normal delivery for the setup — this uses Sam's baseline. + sam [prosody rate 1.35 pitch +3st volume 1.2] (excited): Wait — it actually *worked*! + sam [prosody rate 0.9]: ...which, if you think about it, changes everything. + alex (skeptical): Does it, though? +``` + ## Hard rules (break these and the file won't compile) -1. `podscript: "0.1"` is the first line. +1. A version stamp is the first line — `podscript: "0.2.0"` (the current version). 2. Every speaker appears in `voices:`; every `bed`/`sfx` asset appears in `assets:`. 3. Levels carry `db`; durations carry `s`/`ms`; timecodes carry a unit. No bare numbers. 4. Identifiers (speaker/asset/label names) are ASCII: `[A-Za-z_][A-Za-z0-9_-]*`. @@ -195,7 +270,7 @@ This is the repo's canonical [`examples/cold_open.podscript`](../examples/cold_o ```podscript -podscript: "0.1" +podscript: "0.2.0" # cold_open.podscript — canonical example # diff --git a/docs/SPEC.md b/docs/SPEC.md index 01b9e0f..12bf4dc 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -3,15 +3,17 @@ SPDX-FileCopyrightText: 2026 Mikkel Bergmann SPDX-License-Identifier: CC-BY-4.0 --> -# Podscript Specification — v0.1 +# Podscript Specification — v0.2.0 This is the specification of the Podscript language for declaratively producing podcasts: lexical structure, grammar, AST, semantics (anchors, two-pass resolution, the gain model), the resolved IR schema, and the conformance, determinism, extensibility, and security rules an implementation must follow. -**Status:** v0.1 — first public draft. The grammar and IR are stable; normative -sections (§13–§21) are open for implementer feedback before v1.0. +**Status:** v0.2.0 — the grammar and IR are stable; normative sections (§13–§21) are +open for implementer feedback before v1.0. This version adds the prosody model (§5.5) as +a backward-compatible (minor) addition over v0.1; see [`CHANGELOG.md`](../CHANGELOG.md). +The spec follows [Semantic Versioning](https://semver.org/) (§20). **License:** This specification text is licensed **CC BY 4.0**. A reference implementation, when published, is licensed separately (Apache-2.0). See §21. @@ -65,11 +67,16 @@ those conformance classes individually (§13). The first non-blank, non-comment line of a script MUST be a version declaration: ``` -podscript: "0.1" +podscript: "0.2.0" ``` -An engine MUST refuse a script whose declared major/minor version it does not -implement, with a clear error, rather than attempt a best-effort parse. +The value is a **semantic version** (`MAJOR.MINOR.PATCH`, §20). The `PATCH` component +MAY be omitted and defaults to `0`, so `0.1` and `0.1.0` are equivalent. A script MUST +declare a version that supports every feature it uses (a script using prosody, added in +`0.2.0`, MUST declare at least `0.2.0`); authoring against the current version is always +valid. An engine matches on `MAJOR.MINOR` — `PATCH` differences are always compatible — +and MUST refuse a script whose `MAJOR.MINOR` it does not implement, with a clear error, +rather than attempt a best-effort parse. ### 1.3 Indentation and blocks @@ -100,6 +107,7 @@ IDENT := [A-Za-z_][A-Za-z0-9_-]* NUMBER := -?[0-9]+ ("." [0-9]+)? # decimal point is always "." (locale-independent) DURATION := NUMBER ("s" | "ms") DB := NUMBER "db" # case-insensitive; "-6db", "-18dB" +SEMITONES := ("+" | "-") NUMBER "st" # signed pitch shift, e.g. +2st, -1st TIMECODE := NUMBER ":" NUMBER ("." NUMBER)? # m:ss(.ms), e.g. 0:23, 1:04.5 | DURATION # e.g. 8s, also accepted ``` @@ -165,11 +173,20 @@ voice_field := "voice" ":" IDENT # provider voice name/id | "provider" ":" IDENT # default: project default | "preset" ":" IDENT # named DSP chain (see §7) | "style" ":" IDENT # provider style/emotion hint + | "prosody" ":" prosody_obj # baseline delivery: rate/pitch/volume (§5.5) + +prosody_obj := "{" prosody_kv ("," prosody_kv)* "}" +prosody_kv := "rate" ":" NUMBER # speaking-rate multiplier; 1.0 = natural + | "pitch" ":" SEMITONES # baseline pitch shift, e.g. +2st, -1st + | "volume" ":" NUMBER # vocal-intensity multiplier; 1.0 = natural ``` The `IDENT` before the colon is the **speaker id** used in speech lines and anchors. A voice MAY also carry a `language` field (BCP 47 tag, e.g. `en-US`, -`da-DK`) to select the synthesis language; see §19. +`da-DK`) to select the synthesis language; see §19. The optional `prosody` object +sets the speaker's **baseline delivery** — its habitual `rate`, `pitch`, and +`volume` (§5.5); a per-line `prosody` clause (§5.5) overrides it per attribute. +Prosody is advisory synthesis guidance (§5.5), not a mix or timing control. ### 2.4 `assets` @@ -240,7 +257,8 @@ direction := "(" WORDS ")" # delivery direction → TTS pro - **`cue` `[ … ]`** — timing/engineering clauses (same clause grammar as directives, §5). This is how a speech line carries an anchor without polluting the spoken text. - **`direction` `( … )`** — a human-language delivery note (e.g. `(skeptical)`, - `(warm)`), passed to the TTS engine as a style/prosody hint. Not timing. + `(warm)`), passed to the TTS engine as the emotional/attitudinal prosody hint (§5.5). + Not timing. - **`TEXT`** — the spoken words. This is the *only* freeform text in the language. It is stored as a typed `content` array of inline nodes (§3.1), not a raw string. - **`label`** — optional `#name` to make this clip referenceable by anchors (§6). @@ -257,7 +275,10 @@ markup: {say "lead" as "led"} → a Pronounce node: speak the first word as the second ``` -Anything else is `Text`. Inline markup is optional; plain prose is valid. +Anything else is `Text`. Inline markup is optional; plain prose is valid. Emphasis and +breaks are the inline prosodic controls — word-level stress and phrasing/rhythm +respectively; the continuous prosody attributes (rate, pitch, volume) and the full +prosody model are in §5.5. A **Pronounce** node respells one occurrence of a word for the synthesis engine (`{say "" as ""}`). It overrides the `pronounce` header lexicon @@ -334,7 +355,7 @@ set**: `sfx whoosh after sam gain -3db` ≡ `sfx whoosh gain -3db after sam`. Ea clause *kind* may appear at most once per line (a duplicate is a validation error). ```ebnf -clause := anchor | gain | fade | crossfade | duck | at | loopmode +clause := anchor | gain | fade | crossfade | duck | at | loopmode | prosody anchor := ("after" | "with" | "before") ref offset? ref := IDENT ("." event)? # clip id, speaker, or asset event @@ -347,10 +368,15 @@ crossfade := "crossfade" DURATION ("at" anchor_or_time)? duck := "duck-under" IDENT ("to" DB)? ("attack" DURATION)? ("release" DURATION)? at := "at" TIMECODE # absolute placement (escape hatch) loopmode := "loop" | "once" +prosody := "prosody" prosody_attr+ # one or more delivery attributes (§5.5) +prosody_attr := "rate" NUMBER | "pitch" SEMITONES | "volume" NUMBER anchor_or_time := anchor | TIMECODE ``` +A `prosody` clause is only meaningful on a speech `cue`; it is **advisory synthesis** +guidance, not a timing or mix directive (see §5.5). + ### 5.1 Anchors (`after` / `with` / `before`) | Form | Resolved start/position | @@ -422,6 +448,53 @@ regions whose ramps would intersect are merged so the bed does not bob back up between them. This duck envelope is one of the envelopes combined under §9's `base + min(...)` rule. +### 5.5 Prosody (delivery) + +**Prosody** is the suprasegmental shape of speech — the properties of syllables and +larger units rather than the individual phonemes. Podscript models it along the +dimensions listed below; the three *continuous* ones are set with the `prosody` control +(a voice-level baseline, §2.3, or a per-line `cue` clause), and the rest are expressed +with constructs defined elsewhere in this spec. + +| Prosodic dimension (acoustic correlate) | Podscript control | +|---|---| +| **Rate** — tempo / duration | `prosody rate ` — multiplier, `1.0` = natural | +| **Pitch** — fundamental frequency, baseline intonation | `prosody pitch <±n>st` — semitone shift | +| **Volume** — intensity / vocal effort | `prosody volume ` — multiplier, `1.0` = natural | +| **Stress / focus / emphasis** — prominence | `*word*` emphasis inline (§3.1) | +| **Rhythm / pausing** — phrasing, chunking | `...` and `{break }` (§3.1) | +| **Intonation contour** — sentence type (question / statement) | sentence punctuation in the spoken text | +| **Emotional / attitudinal / voice quality** — timbre, affect | `(direction)` delivery note (§3) | + +`prosody volume` is **vocal intensity** (how forcefully the voice speaks — which also +changes timbre), **not** a mix level: post-synthesis level is the `gain` clause (§5, §9) +in dB. Overall loudness of a finished line is therefore two independent things — how the +voice performs (`volume`) and where its fader sits (`gain`). + +A voice-level `prosody` object (§2.3) sets a speaker's habitual delivery; a per-line +`prosody` clause **overrides it per attribute** (a line that sets only `pitch` keeps the +voice's baseline `rate`). Attributes left unset fall back to the voice baseline, then to +the natural default (`rate 1.0`, `pitch +0st`, `volume 1.0`). + +Prosody is a **synthesis** concern, not a timing one. It changes the speech audio the +provider returns (and therefore a clip's measured duration), but it is **not** an input +to anchor resolution or the gain model: the resolver takes clip durations as supplied +input (§8), so a `prosody` clause never alters `start`/`dur`/`end`. The resolver carries +the resolved **effective** prosody onto the clip (§11.1); the renderer applies each +attribute to the provider's corresponding control, **clamping** to the provider's +supported range and recording the effective (post-clamp) values in the render manifest +(§14.3). Because prosody changes the returned audio, it is part of the synthesis cache +key (§14.3). + +Prosody is **advisory**, and this is a deliberate exception to the fail-closed rule +(§15.4): the three continuous attributes are **core** vocabulary (no `requires`), but an +engine applies only what its provider supports and MAY drop an attribute it cannot honour +(recording what it applied). Unlike a dropped `duck` or `fade`, an unhonoured prosody hint +degrades *expressiveness*, not *mix correctness* — the same best-effort contract as +`(direction)`. Rate is honoured by essentially every engine; pitch and volume vary more. +Podscript models a per-utterance envelope only; finer, within-line variation is authored +by splitting a line into consecutive cues, each with its own prosody. + --- ## 6. Labels and references @@ -504,8 +577,9 @@ interface Script { scenes: Scene[] } interface Requirement { ns: string; optional: boolean } -interface Voice { voice?: string; provider?: string; preset?: string; style?: string; language?: string } +interface Voice { voice?: string; provider?: string; preset?: string; style?: string; language?: string; prosody?: Prosody } interface Asset { path: string; loop?: "loop" | "once" } +interface Prosody { rate?: number; pitch?: number; volume?: number } // pitch in semitones; §5.5 ``` ```ts @@ -541,6 +615,7 @@ type Clause = | { kind: "duck"; track: string; toDb?: number; attackMs?: number; releaseMs?: number } | { kind: "at"; seconds: number } | { kind: "loop"; mode: "loop" | "once" } + | { kind: "prosody"; rate?: number; pitch?: number; volume?: number } // delivery (§5.5); speech cue only; pitch in semitones type Event = "start" | "end" | "fade-in" | "fade-out" type Inline = { kind: "text"; value: string } @@ -572,12 +647,13 @@ and by any future tooling (visual timeline, preview renderer). ```jsonc { - "podscript": "0.1", + "podscript": "0.2.0", "meta": { "title": "…", "lufs": -16, "true_peak": -1 }, "pronounce": { "Seibold": "SY-bold" }, // term → respelling (§2.5); omitted when absent "clips": [ { "id": "c1", "track": "speech", "voice": "sam", "preset": "host", + "prosody": { "rate": 1.1, "pitch": -1 }, "start": 8.0, "dur": 6.5, "end": 14.5, "content": [ { "kind": "text", "value": "…" } ], "anchor": { "rel": "with", "ref": "b1", "event": "fade-out" } @@ -624,6 +700,13 @@ and by any future tooling (visual timeline, preview renderer). the IR root verbatim and omitted when absent. It is applied at synthesis time, so speech clips' `content` retains the original spelling (inline `pronounce` nodes, §3.1, are preserved within `content`); timing and resolution are unaffected. +- **Effective prosody.** A speech clip carries a `prosody` object (§5.5) holding the + resolved delivery attributes, merged **per attribute**: the line's `prosody` clause + value if present, else the speaker's voice-level `prosody` baseline (§2.3). Each + attribute is **omitted at its natural default** (`rate 1.0`, `pitch 0`, `volume 1.0`), + and the whole `prosody` object is omitted when no attribute remains. `pitch` is in + semitones. Prosody is synthesis metadata and does not affect timing, so + `start`/`dur`/`end` are identical whether or not `prosody` is present. - **Bed end.** A `fade-out` to `-inf` sets the bed's `end` to the moment the fade completes. Otherwise a bed ends at `start + asset_duration` (or where explicitly cut). `loop: true` beds have no intrinsic end and MUST be bounded by a fade or cut. @@ -732,7 +815,7 @@ recording everything outside the script that affected the output: ```jsonc { - "podscript": "0.1", + "podscript": "0.2.0", "engine": { "name": "…", "version": "…" }, "voices": { "sam": { "provider": "elevenlabs", "model": "…", "version": "…", "seed": 42 } }, "assets": { "intro": { "sha256": "…", "dur_ms": 30000 } }, @@ -741,8 +824,10 @@ recording everything outside the script that affected the output: ``` Speech SHOULD be cached by a content hash of `(text, voice, provider, model, -version, params)`; a cache hit MUST return byte-identical audio. Given a script plus -its manifest and cache, a re-render MUST reproduce the prior output. +version, prosody, params)`; a cache hit MUST return byte-identical audio. The effective +prosody (§5.5 — `rate`, `pitch`, `volume`) is part of the key because it changes the +returned audio, and its post-clamp values are recorded per voice in the manifest. Given a +script plus its manifest and cache, a re-render MUST reproduce the prior output. --- @@ -866,10 +951,12 @@ form (`X`). ## 20. Versioning and governance -- The spec is versioned with **semantic versioning**. A change that alters the +- The spec is versioned with **[Semantic Versioning](https://semver.org/)** + (`MAJOR.MINOR.PATCH`). **The current version is 0.2.0.** A change that alters the meaning of an existing valid script, or any golden conformance fixture, is a - **major** change. Additive, backward-compatible features are **minor**. - Clarifications are **patch**. + **major** change. Additive, backward-compatible features are **minor** (0.2.0 added + the prosody model, §5.5, over 0.1). Clarifications are **patch**. Every release is + recorded in [`CHANGELOG.md`](../CHANGELOG.md). - Scripts declare the version they target (§1.2); engines refuse versions they do not implement. - Core keyword additions, IR schema changes, and the extension-namespace registry are diff --git a/examples/cold_open.podscript b/examples/cold_open.podscript index 99cd276..b55c171 100644 --- a/examples/cold_open.podscript +++ b/examples/cold_open.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" # cold_open.podscript — canonical example # diff --git a/examples/minimal.podscript b/examples/minimal.podscript index aac1fd3..9848708 100644 --- a/examples/minimal.podscript +++ b/examples/minimal.podscript @@ -1,4 +1,4 @@ -podscript: "0.1" +podscript: "0.2.0" voices: host: { voice: Adam, preset: host } diff --git a/examples/prosody.podscript b/examples/prosody.podscript new file mode 100644 index 0000000..b7c7028 --- /dev/null +++ b/examples/prosody.podscript @@ -0,0 +1,17 @@ +podscript: "0.2.0" + +# prosody.podscript — controlling delivery +# +# Prosody is *how* a line is spoken. Sam has a brisk, slightly low baseline; +# individual lines override rate / pitch / volume. Within-sentence variation is +# authored as consecutive cues, each with its own prosody. + +voices: + sam: { voice: Adam, preset: host, prosody: { rate: 1.1, pitch: -1st } } + alex: { voice: Rachel, preset: warm-host } + +scene demo: + sam: Normal delivery for the setup — this uses Sam's baseline. + sam [prosody rate 1.35 pitch +3st volume 1.2] (excited): Wait — it actually *worked*! + sam [prosody rate 0.9]: ...which, if you think about it, changes everything. + alex (skeptical): Does it, though? diff --git a/llms.txt b/llms.txt index 5d18f06..238be1d 100644 --- a/llms.txt +++ b/llms.txt @@ -5,13 +5,14 @@ ## Docs - [Authoring cheatsheet](docs/AUTHORING.md): compact, self-contained guide to the whole syntax — the file to read before writing a `.podscript`. -- [Specification](docs/SPEC.md): the normative v0.1 spec — grammar, timing model, gain/duck/fade semantics, the IR, determinism, extensibility, security. +- [Specification](docs/SPEC.md): the normative v0.2.0 spec — grammar, timing model, gain/duck/fade semantics, prosody, the IR, determinism, extensibility, security. - [Glossary](docs/GLOSSARY.md): audio production terms (bed, duck, sting, LUFS) and how they map to the language. - [README](README.md): overview and project orientation. ## Examples - [cold_open.podscript](examples/cold_open.podscript): a canonical multi-scene script (intro bed, fade, duck, crossfade, labels, forward anchor). +- [prosody.podscript](examples/prosody.podscript): controlling delivery — per-voice baseline and per-line rate/pitch/volume, with emphasis and direction. ## Conformance diff --git a/openspec/changes/archive/2026-07-03-add-prosody/.openspec.yaml b/openspec/changes/archive/2026-07-03-add-prosody/.openspec.yaml new file mode 100644 index 0000000..8e26fbe --- /dev/null +++ b/openspec/changes/archive/2026-07-03-add-prosody/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-02 diff --git a/openspec/changes/archive/2026-07-03-add-prosody/design.md b/openspec/changes/archive/2026-07-03-add-prosody/design.md new file mode 100644 index 0000000..72e9d15 --- /dev/null +++ b/openspec/changes/archive/2026-07-03-add-prosody/design.md @@ -0,0 +1,89 @@ +## Context + +Podscript already carries fragments of prosody — `(direction)` (emotion), `*emphasis*` +(stress), and breaks (rhythm) — but nothing unifies them and there is no control over rate, +pitch, or intensity. Prosody (linguistics) spans pitch (F0/intonation), duration +(tempo/rhythm), intensity (loudness), plus stress, pausing, and emotional/attitudinal +delivery. Real engines constrain how we can offer it: ElevenLabs parses no SSML and exposes +only a whole-request `speed`; Piper exposes `length_scale`; only espeak-ng honours per-word +``, at robotic quality. So the portable unit is a **per-utterance envelope**. This +repo is the spec + conformance suite (no reference implementation); the change lands as SPEC +edits + fixtures via an OpenSpec change, mirroring the archived `add-pronunciation` change. + +## Goals / Non-Goals + +**Goals:** +- One coherent prosody model that names every prosodic dimension and gives each a concrete + Podscript control — covering the whole space "to an extent," honestly bounded by engines. +- Continuous knobs (`rate`, `pitch`, `volume`) at two granularities: a per-voice baseline + (character consistency) and a per-line override (a beat of delivery). +- Preserve determinism: prosody is synthesis-only and MUST NOT perturb resolution/timing. + +**Non-Goals:** +- No inline per-word prosody spans — unshippable on flagship engines and hostile to + fail-closed (most engines would silently drop them). +- No pitch *contours*/SSML curves, no emphasis *degrees* — possible follow-ups. +- No change to timing math, the gain model, or existing fixtures. + +## Decisions + +**1. One `prosody` control, SSML-shaped, carrying `rate` / `pitch` / `volume`.** +Grouping the three continuous attributes under a single `prosody` keyword (voice object + +line clause) mirrors SSML `` and reads as one delivery unit. +These map onto the three primary acoustic correlates: duration → `rate` (multiplier), +fundamental frequency → `pitch` (semitones, `±Nst`), intensity → `volume` (multiplier). +*Alternative:* three separate top-level clauses. Rejected — noisier and loses the grouping. +*Alternative:* rate only (the earlier `pace` scope). Rejected — the user asked to cover the +full prosodic range. + +**2. Cover the rest of prosody by reframing existing constructs, not new grammar.** +Stress/focus = `*emphasis*`; rhythm/pausing = `...`/`{break}`; intonation contour = sentence +punctuation; emotion/attitude/voice-quality = `(direction)`. §5.5 presents a dimension→control +table so the model is visibly complete while the new grammar stays minimal. + +**3. `volume` is vocal intensity, explicitly distinct from mix `gain`.** +`volume` asks the *voice* to speak more/less forcefully (which changes timbre), at synthesis; +`gain` sets the clip's post-synthesis fader level in dB. Documenting the split prevents the +obvious confusion and lets both appear on one line. +*Alternative:* omit volume, fold loudness into emphasis. Rejected — leaves a named acoustic +correlate uncovered. + +**4. Prosody never enters resolver timing; the resolver bakes the *effective* value per attribute.** +Durations are supplied input, so prosody (which only changes the returned audio/duration) is +carried as clip metadata, like `preset`/`direction`. The resolver merges per attribute +(`line ?? voice`), omits attributes at their natural default, and omits an empty object. +Keeps resolver conformance deterministic (§14.1); a fixture asserts timing is unchanged. + +**5. Prosody is advisory — a deliberate exception to fail-closed (§15.4).** +The attributes are core vocabulary (no `requires`), but an engine applies only what its +provider supports and MAY drop the rest, recording what it applied. Unlike a dropped +`duck`/`fade`, an unhonoured prosody hint degrades expressiveness, not mix correctness — the +same best-effort contract as `(direction)`. Rate is near-universal; pitch/volume vary. + +**6. Prosody joins the synthesis cache key.** +It changes the returned audio, so it is added to the §14.3 `params`; post-clamp effective +values are recorded in the manifest for reproducibility. + +## Risks / Trade-offs + +- **[Within-line variation is not expressible inline]** → Documented pattern: split the + thought into consecutive cues, each with its own prosody, chained by relational timing. +- **[`volume` confused with `gain`]** → Explicit normative distinction (§5.5) + a fixture/ + requirement showing they coexist independently. +- **[Pitch/volume unsupported on flagship engines]** → Advisory contract: engines drop and + record; §14.1 already scopes synthesis as provider-dependent. +- **[Prosody leaking into resolver timing would break determinism]** → Normative synthesis-only; + a fixture asserts IR timing is unchanged by a prosody clause. +- **[Out-of-range values, e.g. `rate 3.0`]** → Parsed as authored; the renderer clamps and + records effective values. The language hard-codes no one engine's range. + +## Migration Plan + +Additive and backward compatible — `prosody` is omitted when absent, so all existing +parse/resolve fixtures and scripts are byte-unchanged. Engines adopt incrementally; until then +a `prosody` value is carried through the IR and the clip is synthesized at natural delivery. + +## Open Questions + +- Follow-ups if wanted: emphasis *degrees* (`**strong**`), pitch *contours*, and a + beat/breath pause vocabulary — deferred so the model can grow coherently later. diff --git a/openspec/changes/archive/2026-07-03-add-prosody/proposal.md b/openspec/changes/archive/2026-07-03-add-prosody/proposal.md new file mode 100644 index 0000000..ed12475 --- /dev/null +++ b/openspec/changes/archive/2026-07-03-add-prosody/proposal.md @@ -0,0 +1,83 @@ +## Why + +Podscript has no unified way to control **prosody** — the delivery of speech on top of the +words. It can hint emotion (`(excited)`), stress a word (`*emphasis*`), and insert pauses +(`...`, `{break 500ms}`), but these are scattered and there is no control over speaking +**rate**, **pitch**, or **vocal intensity**. Natural narration needs the whole prosodic +range, and authors — including LLMs writing scripts — need one coherent model for it. + +Prosody (linguistics) spans pitch (intonation), duration (tempo/rhythm), intensity +(loudness), plus stress, pausing, and emotional/attitudinal delivery. Real TTS engines +constrain *how* we can offer it: ElevenLabs parses **no SSML** and exposes only a +whole-request `speed`; offline engines (Piper `length_scale`, XTTS) expose only global +rate; true per-word prosody exists only in espeak-ng, at robotic quality. So Podscript +models prosody as a **per-utterance, advisory** envelope: portable where engines support +it, gracefully dropped where they don't, and never affecting the deterministic mix. + +## What Changes + +- Introduce a core **prosody** model in the SPEC (`docs/SPEC.md`, §5.5) that names every + prosodic dimension and maps each to a Podscript control: + - Three **continuous** attributes carried by a new `prosody` control — `rate` (tempo + multiplier), `pitch` (semitone shift, `±Nst`), `volume` (vocal-intensity multiplier): + - a **voice-level baseline** object (§2.3): `sam: { voice: Adam, prosody: { rate: 1.1, pitch: -1st } }` + - a **per-line clause** (§5): `sam [prosody rate 1.35 pitch +3st volume 1.2]: ...`, + overriding the baseline **per attribute**. + - The existing constructs, reframed and cross-referenced as the rest of the model: + **stress** = `*emphasis*`, **rhythm/pausing** = `...`/`{break}`, **intonation contour** + = sentence punctuation, **emotion/attitude/voice-quality** = `(direction)`. +- Add a `SEMITONES` token (`±Nst`); define semantics: attributes default to natural + (`rate 1.0`, `pitch +0st`, `volume 1.0`); engines **clamp** and record effective values + in the manifest; `volume` (vocal effort) is explicitly distinct from mix `gain`. +- Prosody is a **synthesis** directive, not a timing one: the resolver bakes the + **effective** (per-attribute merged) prosody onto the IR clip and omits defaults — so + resolution stays deterministic and TTS-independent (§14.1). +- Prosody joins the §14.3 synthesis cache-key `params`. It is **advisory** — a deliberate + exception to fail-closed (§15.4): core vocabulary, but engines drop attributes they can't + honour (like `(direction)`), degrading expressiveness, never mix correctness. +- Add AST/IR types (`Voice.prosody?`, a `Prosody` interface, a `prosody` `Clause` variant, + IR-clip `prosody`) and two conformance fixtures (`parse/prosody`, `resolve/prosody`). +- **Authoring guidance** (`docs/AUTHORING.md`, non-normative): a *Prosody* section with the + dimension→control table, the baseline+override pattern, the split-into-cues pattern for + within-line variation, a recommended `(direction)` vocabulary, and engine portability notes. + +- **Version bump to 0.2.0 (minor).** Prosody is an additive, backward-compatible feature, so + the spec moves from `0.1` to **`0.2.0`** under Semantic Versioning. Adopt three-part + `MAJOR.MINOR.PATCH` stamps (`PATCH` optional, defaults to `0`; `0.1` ≡ `0.1.0` remains valid + input), record the release in a new `CHANGELOG.md`, and bump the spec-version references (SPEC + header/§1.2/§20, README, `llms.txt`). As a one-time pre-1.0 housekeeping step, **normalize + every bundled example and conformance-fixture stamp to `0.2.0`** so the repo uses one + consistent current version (a stamp-only change; no script's meaning changes). + +Not in scope (to keep the grammar bounded): inline per-word prosody spans (unshippable on +flagship engines), pitch *contours*/SSML curves, and emphasis *degrees* — a possible follow-up. + +## Capabilities + +### New Capabilities +- `prosody`: the `prosody` voice-baseline object and per-line clause (`rate`/`pitch`/ + `volume`), the `SEMITONES` token, per-attribute effective-merge into the IR clip (defaults + omitted), inclusion in the synthesis cache key, the advisory/best-effort contract, the + `volume`-vs-`gain` distinction, and the mapping of the remaining prosodic dimensions onto + existing constructs (`*emphasis*`, breaks, punctuation, `(direction)`). Prosody does not + affect timing/resolution. + +### Modified Capabilities + + +## Impact + +- `docs/SPEC.md` — §1.5 `SEMITONES` token, §2.3 `prosody` voice object, §3/§3.1 cross-refs, + §5 clause grammar (`prosody`), §5.5 Prosody model + dimension table, §10 `Voice`/`Prosody`/ + `Clause` types, §11 IR clip example + §11.1 effective-prosody convention, §14.3 cache key. +- `docs/AUTHORING.md` — *Prosody* section, clause-table row, voices field list. +- `examples/prosody.podscript` — canonical runnable example, single-sourced into AUTHORING via a + CI-guarded tagged block and linked from `README.md`. +- `conformance/` — new `parse/prosody` and `resolve/prosody` fixtures; all fixtures restamped + `0.2.0`; README coverage table. +- Versioning — SPEC header/§1.2/§20, `README.md`, `llms.txt` bumped to `0.2.0`; every example and + fixture stamp normalized to `0.2.0`; new top-level `CHANGELOG.md` (SPDX-headed, CC-BY-4.0). +- Consuming engines parse prosody, pass IR prosody through to the provider's rate/pitch/volume + controls (dropping unsupported attributes), and record effective values in the manifest. +- Backward compatible at the language level: `prosody` is omitted when absent and a `0.1` script + parses unchanged; the fixture re-stamp is a stamp-only housekeeping change, not a semantic one. diff --git a/openspec/changes/archive/2026-07-03-add-prosody/specs/prosody/spec.md b/openspec/changes/archive/2026-07-03-add-prosody/specs/prosody/spec.md new file mode 100644 index 0000000..6cb28fa --- /dev/null +++ b/openspec/changes/archive/2026-07-03-add-prosody/specs/prosody/spec.md @@ -0,0 +1,93 @@ +## ADDED Requirements + +### Requirement: Voice-level prosody baseline + +A voice declaration (§2.3) SHALL accept an optional `prosody` object setting the speaker's +**baseline delivery**, with any of three attributes: `rate` (a positive speaking-rate +multiplier, `1.0` = natural), `pitch` (a signed semitone shift written with the `st` unit, +e.g. `+2st`, `-1st`), and `volume` (a positive vocal-intensity multiplier, `1.0` = natural). +It SHALL parse into the AST field `Voice.prosody` (a `Prosody` object with `rate`/`pitch`/ +`volume` numbers, `pitch` in semitones) and SHALL be omitted entirely when absent. + +#### Scenario: Voice prosody parses into the AST +- **WHEN** a script declares `sam: { voice: Adam, prosody: { rate: 1.1, pitch: -1st } }` +- **THEN** the AST voice for `sam` has `prosody` equal to `{ "rate": 1.1, "pitch": -1 }` + +#### Scenario: Absent prosody omits the field +- **WHEN** a voice declares no `prosody` +- **THEN** the AST voice has no `prosody` key + +### Requirement: Line-level prosody clause + +A speech line's cue bracket (§5) SHALL accept a `prosody` clause carrying one or more of the +attributes `rate `, `pitch <±NUMBER>st`, and `volume `, which override the +speaker's voice-level baseline for that line. The clause SHALL parse into the AST as a +`Clause` of kind `prosody` with the present attributes as numbers (`pitch` in semitones). +Like all clauses it SHALL be order-independent and appear at most once per cue. Prosody +SHALL be a **synthesis** directive: it SHALL NOT affect anchor resolution, timing, or the +gain model. + +#### Scenario: Line prosody clause parses into a clause node +- **WHEN** a speech line is written `sam [prosody rate 1.35 pitch +3st volume 1.2]: Wait!` +- **THEN** the line's cue contains a clause `{ "kind": "prosody", "rate": 1.35, "pitch": 3, "volume": 1.2 }` + +#### Scenario: Prosody does not change resolved timing +- **WHEN** two otherwise-identical scripts differ only in a line's `prosody` clause, resolved + against the same supplied durations +- **THEN** every `start`, `dur`, and `end` in the two IRs is identical + +### Requirement: Resolver merges effective prosody per attribute + +The resolver SHALL emit an effective `prosody` object on each speech clip in the IR (§11), +merged **per attribute**: for each of `rate`/`pitch`/`volume`, the line clause's value if +present, otherwise the speaker's voice-level baseline value if present. Each attribute SHALL +be omitted when its effective value is the natural default (`rate 1.0`, `pitch 0`, +`volume 1.0`), and the whole `prosody` object SHALL be omitted when no attribute remains. + +#### Scenario: Line attribute overrides only that attribute of the baseline +- **WHEN** speaker `sam` has voice `prosody: { rate: 1.1, pitch: -1st }` and a line reads + `sam [prosody rate 1.35 pitch +3st volume 1.2]: ...` +- **THEN** that clip's IR `prosody` is `{ "rate": 1.35, "pitch": 3, "volume": 1.2 }` + +#### Scenario: Unset line attributes fall back to the voice baseline +- **WHEN** speaker `sam` has voice `prosody: { rate: 1.1, pitch: -1st }` and a line reads + `sam: ...` with no prosody clause +- **THEN** that clip's IR `prosody` is `{ "rate": 1.1, "pitch": -1 }` + +#### Scenario: Default attributes and empty object are omitted +- **WHEN** a speaker has no voice `prosody` and a line reads `alex [prosody rate 0.9]: ...` +- **THEN** that clip's IR `prosody` is `{ "rate": 0.9 }` (no `pitch`/`volume` keys) + +#### Scenario: Fully-natural clip omits prosody entirely +- **WHEN** a speaker has no voice `prosody` and the line has no prosody clause +- **THEN** that clip's IR has no `prosody` key + +### Requirement: Prosody is advisory and reproducible + +Prosody SHALL be advisory synthesis guidance, a deliberate exception to the fail-closed rule +(§15.4): the `rate`/`pitch`/`volume` attributes are core vocabulary (no `requires`), but a +conforming renderer SHALL apply only what its provider supports, MAY drop an attribute it +cannot honour, and SHALL clamp applied attributes to the provider's range and record the +effective (post-clamp) values in the render manifest (§14.3). Because prosody changes the +synthesized audio, it SHALL be part of the synthesis cache key `params` (§14.3). + +#### Scenario: Prosody is part of the cache key +- **WHEN** two renders of the same clip use different effective `prosody` values +- **THEN** they compute different synthesis cache keys and do not share a cached result + +#### Scenario: Unsupported attribute is dropped, not an error +- **WHEN** a clip requests `pitch +3st` and the provider has no pitch control +- **THEN** the renderer proceeds, omits the pitch shift, and records the applied prosody in + the manifest (it does not raise a `capability` error) + +### Requirement: Volume is vocal intensity, distinct from mix gain + +Prosody `volume` SHALL denote the voice's **vocal intensity/effort** (which also affects +timbre), applied at synthesis, and SHALL be independent of the mix `gain` clause (§5, §9) +that sets a clip's post-synthesis fader level in dB. The two SHALL be expressible together +without conflict. + +#### Scenario: Volume and gain are separate controls +- **WHEN** a script uses `prosody volume 1.2` on a line and a `gain -6db` on a bed +- **THEN** `volume` is recorded on the clip's `prosody` (synthesis) and `gain` remains a + separate mix control; neither is derived from the other diff --git a/openspec/changes/archive/2026-07-03-add-prosody/tasks.md b/openspec/changes/archive/2026-07-03-add-prosody/tasks.md new file mode 100644 index 0000000..6b8dcf2 --- /dev/null +++ b/openspec/changes/archive/2026-07-03-add-prosody/tasks.md @@ -0,0 +1,67 @@ +## 1. SPEC grammar + types (`docs/SPEC.md`) + +- [x] 1.1 Add the `SEMITONES := ("+"|"-") NUMBER "st"` token to §1.5. +- [x] 1.2 Add `| "prosody" ":" prosody_obj` to §2.3 `voice_field`, with `prosody_obj`/ + `prosody_kv` (rate/pitch/volume) and a sentence defining the baseline delivery object. +- [x] 1.3 Add `prosody := "prosody" prosody_attr+` (rate/pitch/volume) to the §5 `clause` + alternation; note it is advisory synthesis (does not affect timing/gain) and overrides + the voice baseline per attribute. (New §5.5 carries the semantics.) +- [x] 1.4 Rewrite §5.5 as **Prosody**: the dimension→control table (rate/pitch/volume + + emphasis/pauses/punctuation/direction), per-attribute merge, synthesis-not-timing, the + advisory/best-effort contract, `volume`-vs-`gain`, and the split-into-cues note. +- [x] 1.5 Add `prosody?: Prosody` to the `Voice` interface, a `Prosody` interface + (`rate?/pitch?/volume?`), and a `{ kind: "prosody"; rate?; pitch?; volume? }` `Clause` + variant (§10); cross-ref §3/§3.1 to §5.5. +- [x] 1.6 Add `"prosody": { ... }` to the §11 speech-clip IR example and a §11.1 convention: + the resolver emits the per-attribute effective prosody, omitting defaults and the empty + object. +- [x] 1.7 Add `prosody` (rate/pitch/volume) to the §14.3 synthesis cache-key `params` and note + post-clamp effective values are recorded in the render manifest. + +## 2. Conformance fixtures (`conformance/`) + +- [x] 2.1 `conformance/parse/prosody/{input.podscript, expected.ast.json}` — a voice with a + `prosody: { rate, pitch }` baseline and per-line `prosody` clauses (incl. `+3st` + semitone parse and a `volume`), alongside inline emphasis, plus a voice/line with no + prosody. +- [x] 2.2 `conformance/resolve/prosody/{input.podscript, durations.json, expected.ir.json}` — + per-attribute merge: baseline inheritance (c1), full override incl. volume (c2), line-only + rate on a voice with no baseline (c3), fully-natural clip with prosody omitted (c4). + Timing chain (`0.0 / 3.25 / 5.3 / 8.15`, 250 ms gap) is independent of prosody. +- [x] 2.3 Update the `conformance/README.md` coverage table with the two new cases. +- [x] 2.4 REUSE headers: new fixtures fall under the existing `conformance/**` aggregate + annotation in `REUSE.toml`; `reuse lint` passes. + +## 3. Authoring guidance (`docs/AUTHORING.md`, non-normative) + +- [x] 3.1 Rewrite the delivery section as **Prosody**: the dimension→control table, the + voice-baseline + per-line-override pattern, and the "split a thought into consecutive + cues" pattern for within-line variation (+ clause-table row and voices field list). +- [x] 3.2 Recommended (non-normative) `(direction)` vocabulary and portability notes on how + `rate`/`pitch`/`volume`, `*emphasis*`, pauses, and punctuation map onto engines like + ElevenLabs (which read tags/punctuation, not SSML). +- [x] 3.3 Add a canonical runnable `examples/prosody.podscript`, single-sourced into AUTHORING + via a `` tagged block (CI-guarded) and referenced from the README. + +## 4. Versioning (semver → 0.2.0) + +- [x] 4.0a Adopt three-part semver in §1.2 (`MAJOR.MINOR.PATCH`, `PATCH` optional/defaults to 0, + match on major/minor) and record the current version + CHANGELOG link in §20. +- [x] 4.0b Bump spec-version references to `0.2.0`: SPEC header/status, §11 IR example (now + carries prosody) and §14.3 manifest example, `README.md`, `llms.txt`, and the AUTHORING + golden-rule/hard-rule stamps. +- [x] 4.0c Normalize **all** example and conformance-fixture stamps to `0.2.0` (prosody scripts, + the non-prosody examples, and every existing golden fixture) so the repo uses one + consistent current version — a stamp-only pre-1.0 housekeeping change. +- [x] 4.0d Add top-level `CHANGELOG.md` (SPDX-headed) with 0.2.0 and 0.1.0 entries. + +## 5. Validation + +- [x] 5.1 Existing parse/resolve fixtures (`cold_open`, `full_episode`, `pronounce`) differ from + `0.1` only by the version stamp (`0.2.0`); AST/IR structure and all timing values are + otherwise unchanged. +- [x] 5.2 Docs-drift guard (`check_doc_examples.py`) passes and `reuse lint` reports full + compliance (incl. the new `CHANGELOG.md`). +- [x] 5.3 New `parse/prosody` and `resolve/prosody` fixtures verified internally consistent + (JSON valid; `0.2.0` stamp; per-attribute merge and default-omission match §5.5/§11.1; + timing recomputed). diff --git a/openspec/specs/prosody/spec.md b/openspec/specs/prosody/spec.md new file mode 100644 index 0000000..ec6c340 --- /dev/null +++ b/openspec/specs/prosody/spec.md @@ -0,0 +1,103 @@ +# prosody Specification + +## Purpose +Define how a Podscript script controls **prosody** — the delivery of speech on top of the words. +The `prosody` control carries the three continuous attributes `rate`, `pitch`, and `volume` as a +per-voice baseline (§2.3) and a per-line `cue` clause (§5), merged per attribute onto the IR clip +(§5.5, §11.1). Prosody is advisory synthesis guidance (a deliberate exception to fail-closed, §15.4) +that joins the synthesis cache key (§14.3) and never affects timing/resolution; the remaining +prosodic dimensions (stress, pausing, intonation contour, emotion) map onto existing constructs +(`*emphasis*`, breaks, punctuation, `(direction)`). + +## Requirements +### Requirement: Voice-level prosody baseline + +A voice declaration (§2.3) SHALL accept an optional `prosody` object setting the speaker's +**baseline delivery**, with any of three attributes: `rate` (a positive speaking-rate +multiplier, `1.0` = natural), `pitch` (a signed semitone shift written with the `st` unit, +e.g. `+2st`, `-1st`), and `volume` (a positive vocal-intensity multiplier, `1.0` = natural). +It SHALL parse into the AST field `Voice.prosody` (a `Prosody` object with `rate`/`pitch`/ +`volume` numbers, `pitch` in semitones) and SHALL be omitted entirely when absent. + +#### Scenario: Voice prosody parses into the AST +- **WHEN** a script declares `sam: { voice: Adam, prosody: { rate: 1.1, pitch: -1st } }` +- **THEN** the AST voice for `sam` has `prosody` equal to `{ "rate": 1.1, "pitch": -1 }` + +#### Scenario: Absent prosody omits the field +- **WHEN** a voice declares no `prosody` +- **THEN** the AST voice has no `prosody` key + +### Requirement: Line-level prosody clause + +A speech line's cue bracket (§5) SHALL accept a `prosody` clause carrying one or more of the +attributes `rate `, `pitch <±NUMBER>st`, and `volume `, which override the +speaker's voice-level baseline for that line. The clause SHALL parse into the AST as a +`Clause` of kind `prosody` with the present attributes as numbers (`pitch` in semitones). +Like all clauses it SHALL be order-independent and appear at most once per cue. Prosody +SHALL be a **synthesis** directive: it SHALL NOT affect anchor resolution, timing, or the +gain model. + +#### Scenario: Line prosody clause parses into a clause node +- **WHEN** a speech line is written `sam [prosody rate 1.35 pitch +3st volume 1.2]: Wait!` +- **THEN** the line's cue contains a clause `{ "kind": "prosody", "rate": 1.35, "pitch": 3, "volume": 1.2 }` + +#### Scenario: Prosody does not change resolved timing +- **WHEN** two otherwise-identical scripts differ only in a line's `prosody` clause, resolved + against the same supplied durations +- **THEN** every `start`, `dur`, and `end` in the two IRs is identical + +### Requirement: Resolver merges effective prosody per attribute + +The resolver SHALL emit an effective `prosody` object on each speech clip in the IR (§11), +merged **per attribute**: for each of `rate`/`pitch`/`volume`, the line clause's value if +present, otherwise the speaker's voice-level baseline value if present. Each attribute SHALL +be omitted when its effective value is the natural default (`rate 1.0`, `pitch 0`, +`volume 1.0`), and the whole `prosody` object SHALL be omitted when no attribute remains. + +#### Scenario: Line attribute overrides only that attribute of the baseline +- **WHEN** speaker `sam` has voice `prosody: { rate: 1.1, pitch: -1st }` and a line reads + `sam [prosody rate 1.35 pitch +3st volume 1.2]: ...` +- **THEN** that clip's IR `prosody` is `{ "rate": 1.35, "pitch": 3, "volume": 1.2 }` + +#### Scenario: Unset line attributes fall back to the voice baseline +- **WHEN** speaker `sam` has voice `prosody: { rate: 1.1, pitch: -1st }` and a line reads + `sam: ...` with no prosody clause +- **THEN** that clip's IR `prosody` is `{ "rate": 1.1, "pitch": -1 }` + +#### Scenario: Default attributes and empty object are omitted +- **WHEN** a speaker has no voice `prosody` and a line reads `alex [prosody rate 0.9]: ...` +- **THEN** that clip's IR `prosody` is `{ "rate": 0.9 }` (no `pitch`/`volume` keys) + +#### Scenario: Fully-natural clip omits prosody entirely +- **WHEN** a speaker has no voice `prosody` and the line has no prosody clause +- **THEN** that clip's IR has no `prosody` key + +### Requirement: Prosody is advisory and reproducible + +Prosody SHALL be advisory synthesis guidance, a deliberate exception to the fail-closed rule +(§15.4): the `rate`/`pitch`/`volume` attributes are core vocabulary (no `requires`), but a +conforming renderer SHALL apply only what its provider supports, MAY drop an attribute it +cannot honour, and SHALL clamp applied attributes to the provider's range and record the +effective (post-clamp) values in the render manifest (§14.3). Because prosody changes the +synthesized audio, it SHALL be part of the synthesis cache key `params` (§14.3). + +#### Scenario: Prosody is part of the cache key +- **WHEN** two renders of the same clip use different effective `prosody` values +- **THEN** they compute different synthesis cache keys and do not share a cached result + +#### Scenario: Unsupported attribute is dropped, not an error +- **WHEN** a clip requests `pitch +3st` and the provider has no pitch control +- **THEN** the renderer proceeds, omits the pitch shift, and records the applied prosody in + the manifest (it does not raise a `capability` error) + +### Requirement: Volume is vocal intensity, distinct from mix gain + +Prosody `volume` SHALL denote the voice's **vocal intensity/effort** (which also affects +timbre), applied at synthesis, and SHALL be independent of the mix `gain` clause (§5, §9) +that sets a clip's post-synthesis fader level in dB. The two SHALL be expressible together +without conflict. + +#### Scenario: Volume and gain are separate controls +- **WHEN** a script uses `prosody volume 1.2` on a line and a `gain -6db` on a bed +- **THEN** `volume` is recorded on the clip's `prosody` (synthesis) and `gain` remains a + separate mix control; neither is derived from the other