Skip to content

feat(whisper): verify GPU backends and add Vulkan fallback - #1982

Open
IdrisGit wants to merge 12 commits into
OpenWhispr:mainfrom
IdrisGit:feat-cuda-support-linux
Open

IdrisGit wants to merge 12 commits into
OpenWhispr:mainfrom
IdrisGit:feat-cuda-support-linux

Conversation

@IdrisGit

@IdrisGit IdrisGit commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add CUDA → Vulkan → CPU fallback for Whisper startup and inference failures.
  • Verify the backend from whisper.cpp startup output before reporting GPU acceleration.
  • Display the actual active CUDA or Vulkan backend in Settings.
  • Require the complete bundled Linux CUDA runtime during installation.
  • Prefer the bundled Linux CUDA libraries over inherited system CUDA paths.
  • Update GPU documentation and localized UI copy.

Release dependency

This PR depends on the corresponding OW-whisper.cpp PR: OpenWhispr/whisper.cpp#7

The whisper.cpp PR must be merged and released first. After its release is published, this PR must update WHISPER_CPP_TAG and the matching SHA-256 digest in src/helpers/whisperCudaManager.js before it can ship.

Validation

  • Confirmed CUDA and Vulkan locally with npm run dev.
  • Passed 74 targeted tests.
  • Passed typecheck, Vite production build, Prettier, and i18n JSON consistency checks.

Follow-ups for #2168 (boseq)

  • Merged main. The fallback events carry fix(whisper): save and show why GPU acceleration fell back to CPU #2317's reasons. The Linux CUDA libs go through main's requiredLibraries, so an installed 0.0.10 pack shows "GPU pack needs re-downloading".
  • When the bundled whisper-server is itself a GPU build (the AUR openwhispr-cuda package), Settings and the Home banner don't offer the pack. A pack failure that lands on that server doesn't say "switched to CPU".
  • A Vulkan pin to a device that's gone is cleared before the backend check. A CUDA → Vulkan fallback updates open GPU cards. There's no re-download notice for a pack that failed and was then removed.
  • Details: feat(whisper): verify GPU backends and add Vulkan fallback #1982 (comment)

Fixes #1415
Fixes #945
Fixes #2168

Related: #712, #126, #531.

boseq added 11 commits October 13, 2026 16:43
Main's requiredLibraries (OpenWhispr#1687) already verifies a pack's companion
libraries after install, and since OpenWhispr#2501 a pack on disk that lacks them is
reported as needing an update. A parallel requiredFiles list skipped that,
so a binary-only whisper.cpp 0.0.10 Linux pack read as never downloaded and
got the first-time offer instead of "GPU pack needs re-downloading". The
libraries are copied by the existing libPattern; the CUDA EULA that
whisper.cpp#7 adds to the zip is no longer copied into the pack folder.
The fallback reason assumed every failed GPU start exited or timed out.
A server that came up on the CPU with no error in its log (only "no GPU
found") was saved as "startup timed out after 30 s". Say that the GPU
backend never initialized instead; a log line naming the cause still wins.
…n the GPU

Distro packages such as the AUR's openwhispr-cuda replace the bundled
whisper-server with a CUDA build. It already transcribes on the GPU, yet
Settings and the Home banner kept offering the pack (OpenWhispr#2168). The server now
says when a start without pack flags comes up on a GPU. The status IPC
reports it, and both offers stand down. The banner re-checks once the launch
pre-warm is up.
The orphan notice ("GPU pack needs re-downloading") is for packs that data
loss removed. A pack that failed here and then vanished, like the CUDA pack
the AUR openwhispr-cuda launcher deletes on every start, isn't one, and the
notice sent users back to a download that had just failed (OpenWhispr#2168).
… tests

Main added the Arabic locale and the GPU card tests after this branch was
cut. Arabic lacked the six per-backend keys, so the card fell back to
English there, and the card tests still expected the generic
"GPU acceleration ready" / "Enable GPU" copy.
A Vulkan pin to a device that is gone (OpenWhispr#1606) makes whisper.cpp start on
the CPU. The new backend check counted that as a Vulkan failure before the
pin could be cleared, so the user stayed on CPU and Retry failed the same
way. Settle the pin first. Also cover the startup CUDA -> Vulkan move and the
Linux library path.
Restarts merge the resolved GPU options over the last start's. A
fallbackToVulkan left out when false kept the old true, so after deleting
the Vulkan pack a CUDA failure started the CPU binary as "Vulkan" and
recorded a Vulkan failure.
The failure was saved, but open GPU cards only noticed through their 5 s
server poll, which switches the backend wording but doesn't re-read the
packs' status. Tell them to re-read at once, as every other GPU change
does, and keep the CPU pop-up for real CPU fallbacks.
- The GPU card's status poll only ends "activating" again. Taking the pack
  and failure state from the server mislabelled a failed pack once a bundled
  GPU build was running, and Remove then deleted the other pack. The
  status-changed broadcast already moves the card after CUDA -> Vulkan.
- A failed pack that has vanished loses its enabled flag at launch, so the
  orphan notice stays away after the upgrade reset clears the failure.
- A GPU pack failure that lands on the bundled server reports the backend
  it actually runs on, and no "switched to CPU" pop-up shows when that is
  a GPU build.
@boseq

boseq commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Hey @IdrisGit, I pushed some commits on top of this. #2168 (the AUR openwhispr-cuda package on Arch's CUDA 13) has the same root cause as #1415, and this PR fixes it, so I wanted it mergeable again. Your commit is untouched: what I added is a merge of main plus ten commits.

  • Merge with main. It conflicted with fix(whisper): save and show why GPU acceleration fell back to CPU #2317, which saves the reason for each GPU fallback. The fallback events now carry both reason and your fallbackBackend, and _fallbackAndRetry keeps the reason extraction.
  • requiredLibraries instead of requiredFiles. Main already has requiredLibraries (fix(whisper): extract MSVC runtime DLLs beside win32 whisper-server #1687, for the Windows MSVC runtime), and since fix(gpu): tell users when an update needs a newer GPU pack #2501 a pack that lacks those libs shows as needing an update.
    • With a separate requiredFiles, an installed 0.0.10 Linux pack read as never downloaded. The card offered a fresh "Enable" instead of "GPU pack needs re-downloading".
    • The CUDA EULA stays in the release zip (Gemini Support? #7 puts it there) but isn't copied into the pack folder anymore. Happy to put it back if you'd rather keep it there.
  • The backend check and the Vulkan device pin (After upgrading to 1.8.3, Windows GPU detection now fails for all previously-working versions of OpenWhispr #1606). A pin to a device that's gone makes whisper.cpp start on the CPU. The check failed Vulkan before the pin could be cleared, so the user stayed on CPU and Retry failed the same way. The pin is now settled first.
  • fallbackToVulkan is always set. Restarts merge it over the last start's options, so leaving it out when false kept an old true after the Vulkan pack was removed.
  • The card's 5 s status poll only ends "activating" again, as on main.
    • Taking the pack and failure state from the server mislabelled a failed pack once a bundled GPU build was running, and Remove then deleted the other pack.
    • A CUDA failure that lands on Vulkan now tells open GPU cards to re-read instead, so Settings still follows the server to the Vulkan pack. There's no CPU pop-up for it.
  • A pack failure that lands on a bundled GPU build reports the backend the server actually runs on, so no "switched to CPU" pop-up shows then either.
  • A reason for a GPU server that came up on the CPU. fix(whisper): save and show why GPU acceleration fell back to CPU #2317's reason code assumed the process had exited or timed out. When the server's log names no cause (only "no GPU found"), the card said "startup timed out after 30 s". Now it says the server started without initializing its GPU backend. A log line that names the cause (e.g. ggml_cuda_init: failed to initialize CUDA: …) still wins.
  • No pack offer when the bundled server is already a GPU build. The AUR package swaps in a CUDA 13 build of whisper-server-linux-x64, and your stderr check reports it as CUDA. Once that server is running, Settings and the Home banner skip the offer. The banner checks again when the startup pre-warm comes up.
  • No "needs re-downloading" notice for a pack that failed and then disappeared. The AUR launcher deletes the pack on every start, so at launch such a pack's enabled flag is dropped, the same as a delete would.
  • Arabic strings for your new keys (Feat/arabic UI localization #1869 added Arabic), and fix(whisper): GPU card follow-ups — hex crash codes, live Retry in Settings, show the pack in use #2335's card tests now expect the per-backend wording.

Each fix has a test that fails without it, with two gaps: the launch-time flag drop runs in main.js startup code that no test covers, and the bundled-GPU fallback report is tested on the startup path but not the mid-transcription one. The CI checks pass locally. I also ran the 0.0.10 Linux binary on Arch (nvidia-utils 615.78, no CUDA 12 anywhere):

  • On its own, it fails on libcudart.so.12.
  • With the CUDA 12.9 libs from NVIDIA's pip wheels next to it and your LD_LIBRARY_PATH, it loads and starts.
  • That machine has no GPU, so it logged ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected. A unit test feeds that same output to the app's start path and checks it falls back with that line as the reason.

This still needs OpenWhispr/whisper.cpp#7 merged and released, then the WHISPER_CPP_TAG and digest bump here. Until then, Linux "Enable CUDA" fails with "missing required libraries", so this can't merge first. #7 conflicts with master too.

@IdrisGit

IdrisGit commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor Author

@boseq I have resolved the conflict in whisper.cpp#7. Thanks for the heads-up about the release dependency. Once OpenWhispr/whisper.cpp#7 is merged and released, I’ll update WHISPER_CPP_TAG and the Linux CUDA artifact digest in this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants