|
1 | 1 | # VoxWatch QA Baseline Manifest |
2 | | -# Version: 1.0 | Date: 2026-03-24 | Coverage: All endpoints, components, and behaviors |
| 2 | +# Version: 1.1 | Date: 2026-03-25 | Coverage: All endpoints, components, and behaviors |
3 | 3 |
|
4 | 4 | This manifest maps EVERY testable element in the VoxWatch system. If something |
5 | 5 | is not listed here, it is not covered in QA. Update this document whenever the |
@@ -227,7 +227,221 @@ gemini (video+images) → openai (images) → anthropic (images) → grok (image |
227 | 227 | 10. SPA catch-all has path traversal guard |
228 | 228 | 11. Dispatch preview requires VoxWatch on port 8892 |
229 | 229 | 12. Config validators run in order (cameras → enabled) |
| 230 | +13. Natural cadence falls back to flat-string TTS if any ffmpeg step fails |
230 | 231 |
|
231 | 232 | --- |
232 | 233 |
|
233 | | -*Generated: 2026-03-24 | Update on any API, component, or config change* |
| 234 | +## Section 9: Natural Cadence System |
| 235 | + |
| 236 | +### Module: voxwatch/speech/natural_cadence.py |
| 237 | + |
| 238 | +| Item | Detail | |
| 239 | +| ------ | -------- | |
| 240 | +| Entry point | `generate_natural_speech(phrases, audio_pipeline, output_path, config, cadence_config=None) -> bool` | |
| 241 | +| Config dataclass | `CadenceConfig` -- 11 fields, built via `CadenceConfig.from_config(config)` | |
| 242 | +| AI response parser | `parse_ai_response(response: str) -> list[str]` | |
| 243 | +| Pause calculator | `determine_pause_duration(phrase, cadence_config) -> float` | |
| 244 | +| Silence generator | `generate_silence(duration, sample_rate, output_path) -> bool` (async, ffmpeg lavfi anullsrc) | |
| 245 | +| Speed variation | `apply_speed_variation(input_path, output_path, speed) -> bool` (async, ffmpeg atempo) | |
| 246 | +| Segment concat | `concatenate_segments(segment_paths, output_path) -> bool` (async, ffmpeg concat demuxer) | |
| 247 | +| Format normalise | `_convert_to_work_format(input_path, output_path) -> bool` (internal, 44.1 kHz 16-bit mono) | |
| 248 | +| Working format | PCM 16-bit signed, 44100 Hz, mono -- all intermediate files | |
| 249 | +| ffmpeg timeout | 30 seconds per subprocess call | |
| 250 | +| Fallback | Returns False; caller falls back to `audio_pipeline.generate_tts` flat string | |
| 251 | +| Temp cleanup | `TemporaryDirectory` always cleaned in `finally` block | |
| 252 | + |
| 253 | +### parse_ai_response() -- Input Format Priority |
| 254 | + |
| 255 | +| Priority | Format | Detection | |
| 256 | +| ---------- | -------- | ----------- | |
| 257 | +| 1 | JSON array inside markdown code block | regex inside code fences, re.DOTALL | |
| 258 | +| 2 | Bare JSON array anywhere in string | regex match, re.DOTALL | |
| 259 | +| 3 | Plain text sentence split | re.split on sentence-ending punctuation | |
| 260 | +| Fallback | Entire response as single phrase | When all strategies yield empty | |
| 261 | + |
| 262 | +### CadenceConfig -- 11 Parameters (config section: speech.natural_cadence) |
| 263 | + |
| 264 | +| Parameter | Type | Default | Config key | |
| 265 | +| --------- | ---- | ------- | ---------- | |
| 266 | +| `min_pause` | float | 0.2s | `speech.natural_cadence.min_pause` | |
| 267 | +| `max_pause` | float | 0.6s | `speech.natural_cadence.max_pause` | |
| 268 | +| `period_pause` | float | 0.5s | `speech.natural_cadence.period_pause` | |
| 269 | +| `ellipsis_pause` | float | 0.7s | `speech.natural_cadence.ellipsis_pause` | |
| 270 | +| `comma_pause` | float | 0.2s | `speech.natural_cadence.comma_pause` | |
| 271 | +| `min_speed` | float | 0.92 | `speech.natural_cadence.min_speed` | |
| 272 | +| `max_speed` | float | 1.08 | `speech.natural_cadence.max_speed` | |
| 273 | +| `speed_variation_enabled` | bool | True | `speech.natural_cadence.speed_variation` | |
| 274 | +| `leading_pause` | float | 0.3s | `speech.natural_cadence.leading_pause` | |
| 275 | +| `trailing_pause` | float | 0.2s | `speech.natural_cadence.trailing_pause` | |
| 276 | +| `postprocess` | bool | True | `speech.natural_cadence.postprocess` | |
| 277 | + |
| 278 | +### Pause Duration Rules (determine_pause_duration) |
| 279 | + |
| 280 | +| Trailing punctuation | Pause used | |
| 281 | +| --------------------- | ---------- | |
| 282 | +| `...` (ellipsis) | `ellipsis_pause` (0.7s default) | |
| 283 | +| `.` `!` `?` | `period_pause` (0.5s default) | |
| 284 | +| `,` `;` `:` | `comma_pause` (0.2s default) | |
| 285 | +| None / other | random uniform in [min_pause, max_pause] | |
| 286 | + |
| 287 | +### Module: voxwatch/speech/postprocess.py |
| 288 | + |
| 289 | +| Item | Detail | |
| 290 | +| ------ | -------- | |
| 291 | +| Entry point | `apply_natural_postprocess(input_path, output_path) -> bool` (async) | |
| 292 | +| Filter chain | `silenceremove` -> `acompressor` (3:1 ratio, -18 dB threshold) -> `loudnorm` (-16 LUFS) | |
| 293 | +| Silence threshold | -50 dB, 0.1s minimum duration (prevents trimming inter-phrase gaps) | |
| 294 | +| Target loudness | -16 LUFS integrated (EBU R128 / ITU-R BS.1770), TP=-1.5, LRA=11 | |
| 295 | +| Output format | PCM 16-bit, 44100 Hz, mono | |
| 296 | +| Invocation | Called lazily from `generate_natural_speech` when `CadenceConfig.postprocess=True` | |
| 297 | +| Failure mode | Non-fatal -- `generate_natural_speech` logs warning and uses unprocessed audio | |
| 298 | + |
| 299 | +### AudioPipeline Integration |
| 300 | + |
| 301 | +| Method | Description | |
| 302 | +| ------- | ----------- | |
| 303 | +| `AudioPipeline.generate_natural_tts(phrases, output_path)` | Calls `generate_natural_speech`; on False return falls back to standard `generate_tts` | |
| 304 | + |
| 305 | +### Config Section: speech.natural_cadence |
| 306 | + |
| 307 | +Sits under the top-level `speech` key in config.yaml. All 11 parameters from CadenceConfig are read here. Missing keys use dataclass defaults -- the section is entirely optional. |
| 308 | + |
| 309 | +### Test Coverage: tests/test_natural_cadence.py |
| 310 | + |
| 311 | +| Test step | What is verified | |
| 312 | +| --------- | ---------------- | |
| 313 | +| 1 -- parse_ai_response | JSON array, JSON code block, plain text sentence split -- all return correct phrase list | |
| 314 | +| 2 -- determine_pause_duration | `.` period_pause, `...` ellipsis_pause, `,` comma_pause, no-punct in [min, max] | |
| 315 | +| 3 -- generate_silence | 0.5s lavfi silence WAV exists and >= 200 bytes | |
| 316 | +| 4 -- apply_speed_variation | atempo 1.05x output exists and >= 200 bytes | |
| 317 | +| 5 -- apply_natural_postprocess | compression + loudnorm output exists and >= 200 bytes | |
| 318 | +| 6 -- Full A/B pipeline | cadence WAV vs flat espeak WAV both generated for listening comparison | |
| 319 | + |
| 320 | +--- |
| 321 | + |
| 322 | +## Section 10: Response Modes System |
| 323 | + |
| 324 | +### Modules |
| 325 | + |
| 326 | +| Module | Exports | |
| 327 | +| ------- | -------- | |
| 328 | +| `voxwatch/modes/mode.py` | `ResponseMode`, `ToneConfig`, `VoiceConfig`, `BehaviorConfig`, `StageConfig` | |
| 329 | +| `voxwatch/modes/loader.py` | `load_modes`, `get_active_mode`, `get_mode_prompt`, `get_mode_template`, `build_ai_vars`, `extract_ai_vars_from_dispatch_json` | |
| 330 | +| `voxwatch/modes/__init__.py` | Re-exports all of the above as the public API | |
| 331 | + |
| 332 | +### ResponseMode Dataclass Hierarchy |
| 333 | + |
| 334 | +``` |
| 335 | +ResponseMode |
| 336 | ++-- id: str |
| 337 | ++-- category: str (core | advanced | novelty | custom) |
| 338 | ++-- name: str |
| 339 | ++-- description: str |
| 340 | ++-- effect: str |
| 341 | ++-- tone: ToneConfig |
| 342 | +| +-- mood: str (default neutral) |
| 343 | +| +-- speed_multiplier: float (default 1.0) |
| 344 | +| +-- radio_effect: bool (default False) |
| 345 | ++-- voice: VoiceConfig |
| 346 | +| +-- kokoro_voice: Optional[str] |
| 347 | +| +-- openai_voice: Optional[str] |
| 348 | +| +-- elevenlabs_voice: Optional[str] |
| 349 | +| +-- piper_model: Optional[str] |
| 350 | ++-- behavior: BehaviorConfig |
| 351 | +| +-- is_dispatch: bool (default False) |
| 352 | +| +-- use_radio_effect: bool (default False) |
| 353 | +| +-- officer_response: bool (default True) |
| 354 | +| +-- json_ai_output: bool (default False) |
| 355 | +| +-- scene_context_prefix: bool (default True) |
| 356 | ++-- stages: dict[str, StageConfig] |
| 357 | + +-- StageConfig |
| 358 | + +-- prompt_modifier: str |
| 359 | + +-- templates: list[str] |
| 360 | +``` |
| 361 | + |
| 362 | +### Built-in Modes (13 total) |
| 363 | + |
| 364 | +| ID | Category | Name | |
| 365 | +| ---- | --------- | ------ | |
| 366 | +| `police_dispatch` | core | Police Dispatch | |
| 367 | +| `live_operator` | core | Live Operator | |
| 368 | +| `private_security` | core | Private Security | |
| 369 | +| `homeowner` | core | Homeowner | |
| 370 | +| `evidence_collection` | core | Evidence Collection | |
| 371 | +| `standard` | core | Standard (fallback) | |
| 372 | +| `silent_pressure` | advanced | Silent Pressure | |
| 373 | +| `neighborhood_alert` | advanced | Neighborhood Alert | |
| 374 | +| `automated_surveillance` | advanced | Automated Surveillance | |
| 375 | +| `mafioso` | novelty | Mafioso | |
| 376 | +| `disappointed_parent` | novelty | Disappointed Parent | |
| 377 | +| `pirate_captain` | novelty | Pirate Captain | |
| 378 | +| `tony_montana` | novelty | Tony Montana | |
| 379 | + |
| 380 | +### loader.py Public API |
| 381 | + |
| 382 | +| Function | Signature | Description | |
| 383 | +| ---------- | ----------- | ------------- | |
| 384 | +| `load_modes` | `(config) -> dict[str, ResponseMode]` | Loads built-ins, merges user-defined modes from `response_modes.modes` | |
| 385 | +| `get_active_mode` | `(config, camera_name=None) -> ResponseMode` | Resolves active mode; honours per-camera overrides; falls back to standard | |
| 386 | +| `get_mode_prompt` | `(mode_def, stage, ai_vars) -> str` | Returns AI system prompt with mode prompt_modifier applied and vars substituted | |
| 387 | +| `get_mode_template` | `(mode_def, stage, ai_vars, index=0) -> str` | Renders fallback template string with {variable} substitution | |
| 388 | +| `build_ai_vars` | `(config, camera_name, ...) -> dict` | Assembles all 8 AI description variables with neutral fallbacks | |
| 389 | +| `extract_ai_vars_from_dispatch_json` | `(ai_json_str) -> dict` | Parses dispatch-mode JSON AI response into AI vars dict | |
| 390 | + |
| 391 | +### Mode Resolution Order (get_active_mode) |
| 392 | + |
| 393 | +1. `response_modes.camera_overrides[camera_name]` -- per-camera override (highest priority) |
| 394 | +2. `response_modes.active_mode` -- global active mode |
| 395 | +3. `response_mode.name` or `persona.name` -- legacy single-key format |
| 396 | +4. standard -- final fallback if mode ID not found in loaded library |
| 397 | + |
| 398 | +### AI Description Variables (8 total) |
| 399 | + |
| 400 | +| Variable | Source | Neutral fallback | |
| 401 | +| --------- | -------- | ---------------- | |
| 402 | +| `{clothing_description}` | AI vision response | "the individual" | |
| 403 | +| `{location_on_property}` | AI vision response | "the property" | |
| 404 | +| `{behavior_description}` | AI vision response | "their current actions" | |
| 405 | +| `{suspect_count}` | AI vision response | "one" | |
| 406 | +| `{address_street}` | `config.property.street` | "this address" | |
| 407 | +| `{address_full}` | `config.property.full_address` | "this address" | |
| 408 | +| `{time_of_day}` | datetime.now().hour at call time | "this hour" | |
| 409 | +| `{camera_name}` | Frigate detection event | "the camera" | |
| 410 | + |
| 411 | +Time-of-day labels: early morning (hours 5-8), morning (9-11), afternoon (12-16), evening (17-20), night (all other hours). |
| 412 | + |
| 413 | +### Per-Camera Mode Overrides (config.yaml) |
| 414 | + |
| 415 | +```yaml |
| 416 | +response_modes: |
| 417 | + active_mode: police_dispatch |
| 418 | + camera_overrides: |
| 419 | + backyard_cam: homeowner |
| 420 | + front_door: police_dispatch |
| 421 | +``` |
| 422 | +
|
| 423 | +Override lookup uses exact camera name string match against `camera_overrides` dict keys. |
| 424 | + |
| 425 | +### Custom Mode Support |
| 426 | + |
| 427 | +User-defined modes in config.yaml under `response_modes.modes` are parsed via `_parse_mode_from_dict`. |
| 428 | +Required field: `id`. Optional: `category` (default custom), name, description, effect, tone.*, voice.*, behavior.*, stage templates. |
| 429 | +Invalid entries are logged and skipped without crashing the service. |
| 430 | + |
| 431 | +### Variable Substitution Safety |
| 432 | + |
| 433 | +`_substitute_vars` uses `_SafeFormatMap` (dict subclass). Unknown {placeholder} tokens returned as-is. Malformed format strings returned unchanged. |
| 434 | + |
| 435 | +### extract_ai_vars_from_dispatch_json |
| 436 | + |
| 437 | +Parses a JSON object string from dispatch-mode AI responses. Field mapping: |
| 438 | +- `description` -> `clothing_description` |
| 439 | +- `location` -> `location_on_property` |
| 440 | +- `suspect_count` -> `suspect_count` |
| 441 | +- `behavior` or `movement` -> `behavior_description` |
| 442 | + |
| 443 | +On JSON parse failure: returns dict with all empty-string values (non-fatal). |
| 444 | + |
| 445 | +--- |
| 446 | + |
| 447 | +*Generated: 2026-03-25 | Update on any API, component, or config change* |
0 commit comments