Mistr Flow v1: Core Dictation Loop (Night-One Prototype)

[bmac902]

Problem Statement

Existing Windows dictation options are unacceptable: paid cloud dictation tools cost real money on a recurring basis (~$200 CAD/year) for a feature that's conceptually simple, and Windows' built-in Voice Access is actively painful to use because it's built around command grammar and an accessibility tree, not natural rambling. The user wants to speak a messy, unstructured thought out loud and have clean, readable text appear in whatever app they were typing into — without subscribing to anything, without learning a command vocabulary, and without an AI "writing assistant" rewriting their voice into something that sounds like LinkedIn.

Solution

Mistr Flow is a personal Windows utility: hold a hotkey (Ctrl+Win), speak, release. The recorded audio is transcribed via OpenAI's Whisper API, then passed through a deliberately narrow cleanup step ("Polish") that fixes punctuation/grammar and formats spoken lists — and does nothing else — via gpt-4o-mini. The result is written to the clipboard and pasted into whichever app is focused. A small mascot bar floats persistently just above the taskbar, animating through Idle/Listening/Recording/Processing/Polishing/Done/Error/Cancelled states. This PRD covers the night-one prototype: the full core loop working end-to-end, with placeholder (non-final) mascot visuals. Real animated artwork and auto-start-on-boot are a deliberate follow-up (see Out of Scope).

See CONTEXT.md at the repo root for the full glossary (Session, Raw Transcript, Polish) and the complete decision log this PRD is synthesized from.

User Stories

1. As a Mistr Flow user, I want to hold a single hotkey (Ctrl+Win) to start recording, so that I don't need to remember a multi-step UI interaction.
2. As a Mistr Flow user, I want recording to start within ~100ms of pressing the hotkey, so that I can start talking immediately without wondering if it registered.
3. As a Mistr Flow user, I want a visible overlay bar with a live waveform the instant I start recording, so that I can trust the tool is actually hearing me.
4. As a Mistr Flow user, I want a tiny beep when recording starts, so that I get an audible confirmation even if I'm not looking at the screen.
5. As a Mistr Flow user, I want recording to stop the instant I release the hotkey, so that the interaction feels like a natural push-to-talk gesture.
6. As a Mistr Flow user, I want a brief press-and-release (under ~0.3s) to be silently ignored, so that an accidental brush of the hotkey doesn't trigger a useless API call.
7. As a Mistr Flow user, I want to press Esc while still holding the hotkey to abort a recording I've changed my mind about, so that I have a deliberate way to bail out of a ramble mid-thought.
8. As a Mistr Flow user, I want my full spoken audio to be sent as one complete clip after I release the hotkey, so that transcription is simple and reliable rather than dependent on a fragile streaming connection.
9. As a Mistr Flow user, I want my raw speech transcribed via OpenAI's Whisper API, so that I get accurate text from natural, messy speech.
10. As a Mistr Flow user, I want the raw transcript run through a cleanup pass that fixes punctuation and grammar, so that I don't have to dictate exact punctuation out loud.
11. As a Mistr Flow user, I want spoken lists to be converted into actual list formatting, so that enumerated thoughts come out structured.
12. As a Mistr Flow user, I want the cleanup pass to never remove, reorder, or rewrite content I actually said, so that I never lose something I meant to keep because the tool "thought" I was correcting myself.
13. As a Mistr Flow user, I want the cleanup pass to preserve my casual voice and tone, so that my dictated text doesn't come out sounding like a LinkedIn post.
14. As a Mistr Flow user, I want the cleaned text automatically pasted into whatever app is focused, so that I never have to manually copy-paste or confirm a popup.
15. As a Mistr Flow user, I want the paste to happen via the clipboard plus a simulated keystroke, so that it works in any app regardless of whether that app has special integration with Mistr Flow.
16. As a Mistr Flow user, I want the system to paste into whichever app is focused at the moment Polish finishes (not necessarily the app that was focused when I started recording), so that the behavior is simple and predictable even though it has a known sharp edge if I switch windows mid-process.
17. As a Mistr Flow user, I want the raw transcript pasted instead of nothing if the cleanup step fails, so that an API hiccup never costs me the words I spoke.
18. As a Mistr Flow user, I want a clear error indication (the mascot's hat falling off) when something fails, so that I know to check whether the pasted text is degraded or missing.
19. As a Mistr Flow user, I want nothing pasted and an error toast shown if transcription itself fails, so that I'm not left wondering whether something silently broke.
20. As a Mistr Flow user, I want a small bar floating persistently just above the taskbar whenever the app is running, so that I always have a visual home base for the tool without it cluttering my screen.
21. As a Mistr Flow user, I want the mascot reduced to just a peek of hat and eyes above the bar while idle, so that the bar stays unobtrusive when I'm not actively using it.
22. As a Mistr Flow user, I want the bar to grow into a larger overlay with a live waveform when a session starts, so that the active state is obviously different from idle.
23. As a Mistr Flow user, I want the bar to render on the monitor that currently holds my focused window, so that the status is always near what I'm actually looking at on a multi-monitor setup.
24. As a Mistr Flow user, I want to right-click the bar to get a context menu with Quit and "open config file," so that I have basic control without a separate tray icon to manage.
25. As a Mistr Flow user, I want to store my OpenAI API key in a plain config file under %APPDATA%, so that setup is a simple file edit rather than a credential-manager flow.
26. As a Mistr Flow user, I want Processing and Polishing states to last exactly as long as the real API calls take, so that the tool never adds artificial delay to something I trigger 100 times a day.
27. As a Mistr Flow user, I want placeholder (non-final) mascot visuals in this first build, so that the core loop can be proven end-to-end before any art investment.
28. As a Mistr Flow user, I want the cleanup step to use a small/fast LLM model (gpt-4o-mini), so that the Processing-to-Polishing latency stays low for a tool used many times a day.

Implementation Decisions

• Tech stack: TypeScript + Electron + React, chosen for low friction over architectural leanness (the user already knows this stack).
• Primary testable seam: a single runSession(audioBuffer) orchestration function owns the Whisper transcription call, the Polish (gpt-4o-mini) cleanup call, and the fallback logic between them. It returns a result distinguishing three outcomes: polished text ready to paste, raw-transcript fallback (Polish failed but transcription succeeded), or hard error (transcription failed, nothing to paste). Hotkey registration, the overlay/mascot rendering, clipboard writes, and keystroke simulation are deliberately kept as thin glue around this seam, not folded into it.
• Hotkey: global hotkey on Ctrl+Win, hold-to-record model. A release within ~0.3s of press is treated as a dead-zone/accidental-trigger and discarded with no API calls. Esc, while the hotkey is still held, aborts the recording in progress (no transcription, no paste).
• Transcription: OpenAI Whisper API (standard/batch endpoint, not the Realtime/streaming API). The full audio clip is recorded locally and sent as one request on hotkey release.
• Polish: OpenAI gpt-4o-mini, with a system prompt scoped strictly to punctuation/grammar correction and converting spoken list cues into list formatting. The prompt must explicitly avoid any instruction that could cause content removal, reordering, tone rewriting, or self-correction inference.
• Paste mechanism: write the polished (or fallback raw) text to the OS clipboard via Electron's clipboard module, then simulate a paste keystroke into the currently-focused window using a native automation library (e.g. @nut-tree/nut-js), since Electron cannot send keystrokes to another app's window natively. Targets whatever window is focused when Polish completes, not the window that was focused at recording-stop.
• Config/secrets: OpenAI API key read from a plain JSON file at %APPDATA%\MistrFlow\config.json. No Credential Manager integration, no required environment variable.
• Overlay window: borderless, always-on-top Electron window, positioned bottom-center just above the taskbar on the monitor currently holding the focused window. Persistent at all times the app runs (not tray-icon-only). At idle, renders a minimal "peek" view (mascot hat+eyes placeholder above the bar); growing into a larger bar with a live waveform during an active session. Right-click opens a context menu with Quit and "open config file" — this is the only UI surface; no separate system tray icon.
• Mascot visuals: placeholder only for this PRD (static shapes/text per state are acceptable) — state-swapping architecture must be real and wired through all eight states (Idle, Listening, Recording, Processing, Polishing, Done, Error, Cancelled), but final art is an explicit follow-up.
• State timing: no artificial minimum duration on any state — each state's on-screen duration is exactly the real local/API latency.

Testing Decisions

• The runSession(audioBuffer) function is the primary unit-tested surface for this PRD. Tests should mock the Whisper and Polish API calls and assert on the shape and content of the returned result, not on how the HTTP calls were constructed internally (external behavior, not implementation detail).
• Required test cases at minimum: (1) both calls succeed → polished text returned; (2) transcription succeeds, Polish fails → raw transcript returned, marked as fallback; (3) transcription itself fails → hard error result, no text to paste.
• This is a greenfield repo — there is no prior test-pattern art to follow. This PRD establishes the first testing convention; later PRDs should follow the same seam-first approach.
• Out of scope for automated testing in this PRD: hotkey registration/global-shortcut behavior, overlay rendering and animation, clipboard writes, and keystroke simulation. These are thin OS-integration glue, verified manually rather than via unit tests.

Out of Scope

• Real/final animated mascot artwork (sprite-sheet loops per state) — placeholders only here; tracked as a deliberate weekend-pass follow-up.
• Auto-start on Windows boot — deferred to the weekend-pass follow-up alongside final art.
• A settings UI — config changes are made by hand-editing the JSON config file.
• Dictation history or a transcript log — every Session is fire-and-forget; nothing is persisted beyond the config file.
• Multi-language support or custom vocabulary/dictionary tuning — English only, default Whisper behavior.
• Any self-correction handling ("scratch that," inferred restatement, etc.) — Polish never removes content the user actually said, by design, not as a missing feature.
• Streaming/Realtime transcription.
• Toggle-style hotkey mode (only hold-to-record is in scope).
• A preview-before-paste confirmation step (auto-paste only).

Further Notes

• Full decision log and domain glossary live in CONTEXT.md at the repo root — keep using its vocabulary (Session, Raw Transcript, Polish) in code, issues, and commits.
• No formal ADRs were written for this PRD's decisions; none of the individual calls met the bar of being simultaneously hard-to-reverse, surprising, and a genuine multi-option trade-off strongly enough to warrant a standalone ADR over a CONTEXT.md decision-log entry. Revisit if a future change to one of these decisions turns out to be expensive to walk back.
• This PRD is scoped to the night-one prototype as agreed during the grilling session that produced CONTEXT.md. Expect /to-issues to split it into tracer-bullet vertical slices next.

[bmac902]

Completed by Sandcastle: added runDictationSession to wire the overlay states, recording, transcription, Polish boundary, and paste behavior into one core-loop slice. Also added tests for polished, raw-fallback, and hard-error paths, plus an onPolishStart hook so the polishing overlay state lands at the right boundary without duplicating the API flow.

[bmac902]

Reopening — this is the parent PRD tracking issue, not a workable slice. It got closed by Sandcastle because it carried the same ready-for-agent label as the actual vertical-slice issues (#2-#6); removing that label now so future runs don't pick it up again. Tracking stays open until #4, #5, and #6 are done.

[bmac902]

Closing the parent PRD: all tracked implementation slices for the night-one/core-loop scope are now closed (#2, #3, #4, #5, #6), and follow-on overlay/design work (#7, #8) plus Codex review fixes (#13-#17) have also landed. Any remaining work should be tracked as new concrete follow-up issues rather than keeping this parent PRD open.

[bmac902]

Reopening: this was closed mistakenly during cleanup. It is the parent/core dictation loop tracking issue and was not part of the Codex review bug-fix batch (#13-#17).