Skip to content

feat: add VT Code + Kimi K2.6 eval results (33%)#83

Open
vinhnx wants to merge 2 commits into
vercel:mainfrom
vinhnx:main
Open

feat: add VT Code + Kimi K2.6 eval results (33%)#83
vinhnx wants to merge 2 commits into
vercel:mainfrom
vinhnx:main

Conversation

@vinhnx

@vinhnx vinhnx commented May 31, 2026

Copy link
Copy Markdown

Summary

Adds VT Code agent eval results using Kimi K2.6 via HuggingFace.

Results: 8/24 passed (33%)

Passed evals

  • agent-021-avoid-fetch-in-effect
  • agent-023-avoid-getserversideprops
  • agent-024-avoid-redundant-usestate
  • agent-025-prefer-next-link
  • agent-033-forbidden-auth
  • agent-034-async-cookies
  • agent-038-refresh-settings
  • agent-042-enable-ppr

Changes

  • scripts/run-vtcode-evals.ts — standalone eval runner for vtcode
  • experiments/vtcode.ts — experiment config
  • results.html — visual results page
  • scripts/export-results.ts — added vtcode model names

Agent config

  • Model: moonshotai/Kimi-K2.6:novita via HuggingFace
  • Orchestration: single mode (no planner)
  • Per-turn tool limit: 50
  • Max turns: 100
  • Reasoning: disabled

@vinhnx vinhnx changed the title feat: add vtcode + Kimi K2.6 eval results (33%) feat: add VT Code + Kimi K2.6 eval results (33%) May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant