chore: default scheduled-task model to Kimi K3 (moonshotai/kimi-k3) - #155
Conversation
Switch DEFAULT_TASK_MODEL from anthropic/claude-sonnet-5 to moonshotai/kimi-k3 for headless scheduled customer tasks (customer-prompt-task), for cost/latency. Preserves the anti-hallucination rationale in a comment: Sonnet 5 was originally chosen because it respects the data-grounding rule where haiku-4.5 did not (OneRPM/Apache benchmark, recoupable/chat#1833). Kimi K3's grounding behavior in headless runs is NOT yet re-verified on that benchmark — flagged inline and in the PR to verify before merge. Updates the model-default unit test to assert the new value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Warning Review limit reached
Next review available in: 52 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. 📝 WalkthroughWalkthroughThe default task model changes from ChangesDefault task model
Estimated code review effort: 1 (Trivial) | ~2 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/consts.ts (1)
6-11: 🎯 Functional Correctness | 🔵 TrivialVerify Kimi K3 grounding before enabling this fallback in production.
customerPromptTaskusesDEFAULT_TASK_MODELwheneverscheduled_actions.modelis missing or null, so this change affects real scheduled briefings. The test verifies model routing only; run thechat#1833grounding benchmark and a real scheduled-briefing spot-check before merge, or retain Sonnet 5 until the result is confirmed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/consts.ts` around lines 6 - 11, Before enabling DEFAULT_TASK_MODEL as moonshotai/kimi-k3, run the chat#1833 grounding benchmark and perform a real scheduled-briefing spot-check covering the missing or null scheduled_actions.model fallback. Retain anthropic/claude-sonnet-5 in DEFAULT_TASK_MODEL until Kimi K3 is confirmed not to fabricate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/consts.ts`:
- Around line 6-11: Before enabling DEFAULT_TASK_MODEL as moonshotai/kimi-k3,
run the chat#1833 grounding benchmark and perform a real scheduled-briefing
spot-check covering the missing or null scheduled_actions.model fallback. Retain
anthropic/claude-sonnet-5 in DEFAULT_TASK_MODEL until Kimi K3 is confirmed not
to fabricate.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e7b732fa-4c64-4771-8d45-2bb2c3889aa7
📒 Files selected for processing (2)
src/consts.tssrc/tasks/__tests__/customerPromptTask.test.ts
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/consts.ts">
<violation number="1" location="src/consts.ts:11">
P1: Scheduled customer briefings now use Kimi K3 for the common null-model case before its headless grounding has been verified, so a model regression can send fabricated metrics through the server-side email workflow. Keeping Sonnet 5 as the default until the chat#1833 benchmark passes would preserve the existing anti-hallucination safeguard.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| // (recoupable/chat#1833). Switching the default to Kimi K3 (cost/latency): its data-grounding | ||
| // behavior in headless runs has NOT yet been re-verified on that benchmark — re-run it before | ||
| // relying on this in production, and revert to anthropic/claude-sonnet-5 if it fabricates. | ||
| export const DEFAULT_TASK_MODEL = "moonshotai/kimi-k3"; |
There was a problem hiding this comment.
P1: Scheduled customer briefings now use Kimi K3 for the common null-model case before its headless grounding has been verified, so a model regression can send fabricated metrics through the server-side email workflow. Keeping Sonnet 5 as the default until the chat#1833 benchmark passes would preserve the existing anti-hallucination safeguard.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/consts.ts, line 11:
<comment>Scheduled customer briefings now use Kimi K3 for the common null-model case before its headless grounding has been verified, so a model regression can send fabricated metrics through the server-side email workflow. Keeping Sonnet 5 as the default until the chat#1833 benchmark passes would preserve the existing anti-hallucination safeguard.</comment>
<file context>
@@ -3,6 +3,9 @@ export const RECOUP_API_KEY = process.env.RECOUP_API_KEY;
+// (recoupable/chat#1833). Switching the default to Kimi K3 (cost/latency): its data-grounding
+// behavior in headless runs has NOT yet been re-verified on that benchmark — re-run it before
+// relying on this in production, and revert to anthropic/claude-sonnet-5 if it fabricates.
+export const DEFAULT_TASK_MODEL = "moonshotai/kimi-k3";
</file context>
| export const DEFAULT_TASK_MODEL = "moonshotai/kimi-k3"; | |
| export const DEFAULT_TASK_MODEL = "anthropic/claude-sonnet-5"; |
1 participant
What
Change
DEFAULT_TASK_MODEL(src/consts.ts:8) fromanthropic/claude-sonnet-5→moonshotai/kimi-k3.This is the model used by
customer-prompt-taskfor every scheduled customer briefing whosescheduled_actions.modelcolumn isnull(the common case — e.g. the "Weekly Artist Briefing" actions). The resolution istaskConfig?.model ?? DEFAULT_TASK_MODELinsrc/tasks/customerPromptTask.ts:51.Why
Cost/latency: a single weekly briefing generation on Sonnet 5 cost ~106 credits (~2.6M input tokens). Kimi K3 is materially cheaper for the same headless report workload.
Sonnet 5 was deliberately chosen here as an anti-hallucination safeguard. These tasks run headless — no human to catch a fabricated report before it emails a customer. Sonnet 5 respects the data-grounding rule (refuses to invent metrics) where the prior default (haiku-4.5) did not, verified on the OneRPM/Apache benchmark (recoupable/chat#1833).
Kimi K3's grounding behavior in headless runs has not been re-verified on that benchmark. If it fabricates numbers, customers receive false analytics. Recommend running the chat#1833 benchmark against
moonshotai/kimi-k3before merging, and reverting toanthropic/claude-sonnet-5if it fails. Rationale is preserved inline inconsts.ts.Changes
src/consts.ts— flip default + update the rationale comment to flag the unverified grounding risk.src/tasks/__tests__/customerPromptTask.test.ts— assert the new default (2 passedlocally).Test plan
npx vitest run src/tasks/__tests__/customerPromptTask.test.ts→ 2 passedmoonshotai/kimi-k3🤖 Generated with Claude Code
Summary by cubic
Switch the default model for headless scheduled customer briefings from
anthropic/claude-sonnet-5tomoonshotai/kimi-k3to reduce cost and latency when the task config omits a model. Updated the model-default unit test; removed the Sonnet-5 grounding note fromsrc/consts.ts(K3 grounding still unverified).moonshotai/kimi-k3; revert toanthropic/claude-sonnet-5if it fabricates metrics.Written for commit 515cd25. Summary will update on new commits.
Summary by CodeRabbit
New Features
Tests