Excita: microWakeWord and nanoWakeWord engines - #218
Conversation
Two new first-class engines behind the WakeWordEngine protocol, per the
spec 0011 extension and ADR-0020..0023:
- microwakeword: score (tflite-runtime over stored clips) + package
(tflite_micro); load/feed/train raise NotSupportedError with honest
messages (detection lives on the ESP32; train waits on the worker).
- nanowakeword: load/feed/score/package via NanoInterpreter, residual-
buffered detector matching the openWakeWord adapter shape; train
points at EXCITA_TRAIN_WORKER_URL.
- GET /engines capability matrix; capability gaps return structured 501
{code: engine_capability_missing, engine, capability, message} on
every dispatching route, declared in OpenAPI.
- Model data model: models table with UNIQUE(phrase_id, engine, version),
source upload|filesystem, filesystem_path, metrics envelope+raw;
phrases gain notes/deleted_at via additive migration.
- POST /models/import (artifact + metadata sidecar) and a filesystem
scanner over EXCITA_MODEL_IMPORT_DIR (boot, SIGHUP, POST /models/scan):
new files import, missing files soft-delete, version bumps mint new
rows, unchanged re-scans are no-ops; DELETE on filesystem rows is 409
filesystem_imported_read_only (ADR-0021).
- Deploy targets (file, http_push with X-Excita-* headers,
linked_service_config) with publish-as-row-change semantics; OWW
package(onnx) so all three transports work for every real engine.
- POST /debug/score and POST /train wired through the same 501 contract.
Tests at the TestClient(create_app) seam: capability matrix cells,
phrase-across-engines, sidecar round-trip, scanner lifecycle, deploy
publish (real local HTTP server for header capture), plus artifact-gated
uWW/nWW adapter tests. Spec 0011 header + open questions updated.
- 501 capability-gap bodies are built from authored reason sentences (engines.base.gap_reason) instead of str(exception), so no exception internals can reach a response body (CodeQL py/stack-trace-exposure); the caught error is logged server-side with engine/capability context. - Adapters raise NotSupportedError(gap_reason(...)) so their messages and the HTTP bodies share one source of truth. - Backend protocol methods use explicit raise NotImplementedError stubs instead of bare '...' (CodeQL statement-has-no-effect). - Tests: no side-effecting calls inside assert expressions.
|
Addressed all review comments in a6c33ae: CodeQL — py/stack-trace-exposure (app.py code-quality — assert with side effect (test_app.py): the DELETE (and, proactively, the code-quality — statement has no effect (backend.py Backend protocol): all protocol methods now use explicit Full suite: 53 passed, 2 artifact-gated skips; ruff E/F/W clean. |
Closes #213
What changed?
Excita gains two first-class engines behind the existing
WakeWordEngineprotocol, per the spec in #213 and ADR-0020..0023:Engines
services/excita/engines/microwakeword.py—score(tflite-runtime over stored clips) +package(tflite_micro);load/feed/trainraiseNotSupportedErrorwith honest messages (detection lives on the ESP32; training waits on theEXCITA_TRAIN_WORKER_URLworker spec).services/excita/engines/nanowakeword.py— realNanoInterpreteradapter: 16 kHz mono int16, 1280-sample chunks, residual buffering,reset()clears residual + hidden state;package(onnx);train→NotSupportedErrorpointing at the train worker.GET /enginesreturns the matrix; every capability gap translates to a structured 501 body{code: "engine_capability_missing", engine, capability, message}on all dispatching routes (/detectors,/debug/score,/train, deploy publish), declared in OpenAPI.Data model + import
modelstable:UNIQUE(phrase_id, engine, version),engine_phrase_key,source(upload|filesystem),filesystem_path, metrics{envelope, raw}. Additive migration only — existing openWakeWord installs keep working (Excita: microWakeWord and nanoWakeWord engines #213 story 20). Phrases gainnotes/deleted_atper ADR-0022.POST /models/import: artifact upload + JSON metadata; unknown phrases are created; an.excita.jsonsidecar is written next to the stored artifact so copying it into a mount promotes it without a rewrite (story 15).EXCITA_MODEL_IMPORT_DIR(boot scan, SIGHUP,POST /models/scan): new files import, missing files soft-delete, sidecar version bumps mint new rows (history preserved), unchanged re-scans are no-ops, malformed sidecars skip cleanly.DELETE /models/{id}on filesystem rows → 409filesystem_imported_read_only(ADR-0021).Deploy targets
file,http_push,linked_service_config) work for every engine's native package; publishing is a single row change and a failed push never rolls it back.http_pushsendsX-Excita-Engine/Phrase/Version.package(onnx)implemented (file bytes), matching ADR-0020's matrix.Dependencies:
tflite-runtime>=2.14(linux marker — no macOS arm64 wheels; the adapter raises honestly there) andnanowakewordadded to requirements.txt. One image, one behavior.Why?
Operators with µWW-on-ESP32 or nanoWakeWord fleets had no way to register, compare, score, package, or deploy those models from Excita — the wake-word ops surface only reached openWakeWord. This extends spec 0011's engine roster additively; phrases stay engine-agnostic so cross-engine comparison works (stories 7–8).
How was it tested?
TestClient(create_app(config))): 53 passed, 2 skipped locally./phrases/{id}+/models?phrase_id=; sidecar round-trip; scanner lifecycle (discover / version bump → new row / no-change stability / missing file → soft delete / malformed sidecar skip); boot scan; DELETE refusal; deploy publish tofileand to a real local HTTP server capturingX-Excita-*headers; failed-push-keeps-selection; OWW/debug/scoreend-to-end against the fetched hey_jarvis fixture.score()unit test and nWW arm→feed→wake e2e test are artifact-gated (skip-guard pattern per_wake_models_available()).Breaking changes?
None. All new surface is additive; existing routes unchanged except
POST /detectors's 501 body, which is now structured JSON (previously a plain message string indetail) — the status code is identical.Documentation
services/excita/README.md: engine matrix, model import paths, deploy targets, env reference.docs/specs/0011-excita-wake-word-ops.md: header marks it extended by Excita: microWakeWord and nanoWakeWord engines #213; open-questions training-compute item points at Excita: microWakeWord and nanoWakeWord engines #213 for the deferred parts.