Skip to content

Add Cerebras fast rewording provider#173

Open
MyButtermilk wants to merge 1 commit into
DevEmperor:mainfrom
MyButtermilk:codex/issue-171-cerebras-gemma
Open

Add Cerebras fast rewording provider#173
MyButtermilk wants to merge 1 commit into
DevEmperor:mainfrom
MyButtermilk:codex/issue-171-cerebras-gemma

Conversation

@MyButtermilk

Copy link
Copy Markdown
Contributor

Summary

This PR implements the practical path suggested by MyButtermilk in #171: add a very fast built-in provider option for post-dictation cleanup/rewording instead of immediately expanding every prompt with its own provider/model routing.

Changes:

  • Add Cerebras as a built-in chat-only provider preset.
  • Default Cerebras rewording/chat requests to gemma-4-31b.
  • Expose a small curated Cerebras chat model list: gemma-4-31b, gpt-oss-120b, and zai-glm-4.7.
  • Keep Cerebras out of transcription flows by marking it CHAT_ONLY.
  • Reuse the existing OpenAI-compatible client via https://api.cerebras.ai/v1/.
  • Add a focused registry test to prevent the preset from regressing.

Refs #171.

Why this is valuable

Issue #171 discusses per-prompt provider/model selection for rewording prompts. MyButtermilk's comment points out that the largest immediate UX win may not require full per-prompt routing: common post-dictation cleanup tasks are short, repetitive, and latency-sensitive. Punctuation cleanup, filler-word removal, and light grammar correction benefit most from a provider/model path that starts quickly and returns quickly.

Cerebras is a good fit for that narrow path because its inference API is OpenAI-compatible, so Dictate can reuse the existing provider infrastructure instead of adding new request plumbing. That keeps this change small while still giving users a fast rewording target. The selected default, gemma-4-31b, follows the model suggested in the issue discussion and is listed in Cerebras' model catalog.

This improves the app in a few concrete ways:

  • Users get a dedicated low-latency cleanup provider without configuring a custom endpoint by hand.
  • The existing provider selection UI can surface Cerebras automatically through the registry.
  • Transcription provider lists stay correct because Cerebras is registered as chat-only.
  • The implementation avoids prompt/profile migrations and avoids increasing the complexity of the prompt editor.
  • Future per-prompt routing can still build on this provider preset if the larger feature is implemented later.

Intentional scope

This PR intentionally does not implement full per-prompt provider/model selection. It also does not add provider-specific temperature, max-token, or streaming controls yet. Cerebras supports streaming, but Dictate's current provider preset model is focused on provider capabilities and model defaults, so adding streaming/parameter policy would be a separate UI/configuration change.

Keeping this PR focused makes it easier to review and gives users the main performance-oriented option requested in the issue discussion without expanding the data model.

Validation

Ran:

./gradlew.bat :app:testDebugUnitTest --tests "dev.patrickgold.florisboard.dictate.ProviderRegistryTest" :app:compileDebugKotlin :lib:dictate-core:compileDebugKotlin :wear:compileDebugKotlin

Result: build successful, including ProviderRegistryTest > cerebrasIsAvailableAsFastRewordingProvider().

Also ran git diff --check successfully.

References

@MyButtermilk
MyButtermilk marked this pull request as ready for review July 9, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant