feat(transcription): OpenAI-compatible cloud ASR engine (from #349) - #401
Merged
Conversation
Add an `openai-asr` transcription engine that POSTs audio to any OpenAI-compatible /audio/transcriptions endpoint (stdlib urllib only: verbose_json with a plain-text fallback, and a NoRedirectHandler that blocks credential leaks to cross-origin redirects). Return shape matches the parakeet/whisper backends so the pipeline is unchanged. Security: the API key is read from STENOAI_OAI_API_KEY (env only), mirroring get_cloud_api_key, and is NEVER persisted to config.json. Only the non-secret api_url/model are stored. New CLI commands get-openai-asr-config / set-openai-asr-config handle url/model only — no key is ever accepted via argv. Co-authored-by: Vassista <vassista@icloud.com>
…tion env Mirror the cloud summariser key pattern for the OpenAI ASR key: encrypted at rest under getUserDataDir() (.openai-asr-api-key, honouring the e2e data-dir override), migrated forward from the legacy path, and NEVER written to config.json or argv. It is decrypted only to inject STENOAI_OAI_API_KEY into the transcription subprocess env — both the batch process-streaming job and the live transcribe-stream sidecar (getTranscriptionEnv). Add IPC handlers: get/set-openai-asr-config (non-secret url/model, shells to the CLI like set-cloud-api-url) with api_key_set overridden by the safeStorage truth (hasOpenAiAsrKey), and set-openai-asr-key (saves, or clears on empty string). Teach loadTranscriptionEngine/Context about the new engine, and redact set-openai-asr-config args from the diagnostics echo (the url may embed credentials; the key never travels via argv). Co-authored-by: Vassista <vassista@icloud.com>
Add the `openaiAsr` preload bridge (getConfig/setConfig/setKey), its typed mirror in ipc.ts (+ the 'openai-asr' TranscriptionEngine member), and the useOpenAiAsrConfig / useSetOpenAiAsrConfig / useSetOpenAiAsrKey hooks. In AiTab's engine picker, add a "Cloud API" option gated behind a privacy ConfirmDialog (audio leaves the device), and render an OpenAI config sub-panel: API base URL + model (save-on-blur) and an API-key input that shows only a set/not-set state (via api_key_set, never the value) with save-on-blur and a clear affordance. The section intro copy becomes engine-aware so the on-device privacy promise isn't shown for the cloud engine. Keep the e2e mock IPC contract in sync. Co-authored-by: Vassista <vassista@icloud.com>
Drive the openaiAsr preload bridge to round-trip the non-secret url/model through config.json, and assert the API key is stored encrypted (safeStorage) under the temp dir — the plaintext key never appears in config.json — and that clearing removes the file and flips api_key_set back to false. Model-free: no real ASR endpoint is ever contacted. Skips loudly where safeStorage is unavailable (headless runner), mirroring the cloud-key guard in ai-provider.t2. Co-authored-by: Vassista <vassista@icloud.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 11 files
You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
… reset Address cubic review on the cloud ASR PR: - transcriber: don't require bundled local ASR (parakeet/whisper) imports when the user selected openai-asr. Cloud is a pure-Python urllib REST call and needs no local model, so the ImportError guard now runs only for the on-device engines. - transcriber: fix diarised speaker ordering for text-only (non-verbose_json) endpoints. Those return no per-segment timestamps (a synthetic start=end=0 whole-channel segment), so sorting by start always emitted [You] before [Others]. Now only sort when a channel carries real timing, keep timeless segments in insertion order after any timed content, and omit the fabricated [00:00] timestamp for timeless turns rather than implying a chronology. - config: get_transcription_engine docstring now documents openai-asr. - config: set_openai_asr_api_url rejects blank/whitespace URLs (mirrors set_openai_asr_model), keeping the prior value instead of persisting an unusable empty base URL. - AiTab: clearing the API URL / model field on blur now resets to the default (persisted) rather than leaving a stale value that returns on refresh. - tests: cover the text-only diarised ordering and blank URL/model rejection. Co-authored-by: Vassista <vassista@icloud.com>
Collaborator
|
Heads-up: my review of this PR was finalised just as it merged, so the findings went into issues instead - #403 carries two I'd treat as release-blocking for the feature (meetings over ~13 min fail against the 25 MB upload limit; unvalidated 200 responses can be mistaken for silence and delete the only audio when Save recordings is off), and #404 covers the privacy-docs contradiction. The safeStorage key handling here is exemplary - the issues are about the request path, not the credential story. |
ruzin
added a commit
that referenced
this pull request
Jul 23, 2026
An empty config.json.lock (a runtime lock from Python config writes) was accidentally committed in #401. Remove it and gitignore *.json.lock so it can't be re-added. Co-authored-by: ruzin <ruzin113@icloud.com>
21 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an OpenAI-compatible cloud transcription engine: alongside Parakeet and Whisper, you can point Steno at any OpenAI-compatible
/audio/transcriptionsendpoint (API URL + model + key). Extracted from @Vassista's #349, adapted to the current design, with the API-key storage reworked to match the app's safeStorage convention.Security fix vs. the original PR
#349 wrote the ASR API key plaintext into
config.json. This PR does not. The key is handled exactly like the cloud-summariser key:safeStoragein a.openai-asr-api-keyfile under the user-data dir (saveOpenAiAsrKey/loadOpenAiAsrKey/hasOpenAiAsrKey, with legacy-file migration).STENOAI_OAI_API_KEY(newgetTranscriptionEnv(), applied at both the batch and live-sidecar spawn sites).get_openai_asr_api_key()→os.environ); there is no plaintext setter and no CLI path that takes the key as an argument.config.jsononly ever holds the non-secret URL + model.api_key_set, overridden byhasOpenAiAsrKey()); the value is never returned to the renderer. Clearing removes the file.Other hardening (ported)
_run_openai_asrblocks cross-origin redirects (NoRedirectHandler) so the Bearer token can't leak to a redirected host;verbose_json→textfallback.UI
Re-homed onto the current AiTab engine picker (the PR's
TranscriptionTab.tsxno longer exists) — a "Cloud API" option behind a privacy confirm dialog, plus an API URL / model / key sub-panel using the currentSettingRowprimitives. No settings-layout change.Tests
e2e/specs/cloud-asr-config.t2.spec.ts: sets/gets the non-secret config (persists toconfig.json) and asserts the key flipsapi_key_settrue without the key value ever landing inconfig.json. Model-free (no network/real ASR).typecheck:renderer,ipc-contract(7/7), and the Python unittest suite green locally. The new T2 needs the freshly-built backend bundle (new CLI commands), which CI builds — it validates there.Credit: @Vassista (#349).
Summary by cubic
Adds an OpenAI-compatible cloud transcription engine (
openai-asr) so you can send audio to any/audio/transcriptionsendpoint, with safe, encrypted key storage and a simple settings UI. Also fixes diarised ordering for text-only endpoints and lets cloud ASR run without local model imports.New Features
openai-asrengine alongside Parakeet and Whisper; Python posts audio to/audio/transcriptionswithverbose_jsonthentextfallback and same-origin redirect protection..openai-asr-api-key, injected asSTENOAI_OAI_API_KEY; never inconfig.jsonor argv.api_key_set.get-openai-asr-config,set-openai-asr-config,set-openai-asr-key; diagnostics redactset-openai-asr-configargs.e2e/specs/cloud-asr-config.t2.spec.tsverifies config round-trip and encrypted key storage.Bug Fixes
[00:00]timestamps when no real timing exists.Written for commit da1fd68. Summary will update on new commits.