Skip to content

Commit 763865d

Browse files
authored
Merge pull request #1144 from volcengine/fix/studio-migration-analysis-protocol
Fix/studio migration analysis protocol
2 parents 587d9fe + 40abd6d commit 763865d

65 files changed

Lines changed: 20517 additions & 634 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎frontend/README.md‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -600,9 +600,22 @@ See [deployment and operation](service/studio_release_notifier/README.md).
600600
the framework, entry point, and open questions. Structured frameworks run the
601601
preinstalled `ak migrate`; Dify and Any projects run
602602
`ak migrate --execution in-place` with Codex in the same Session. Evaluation
603-
deploys a temporary Runtime, checkpoints per-case execution as JSONL, judges
604-
batches in one fresh resumable Codex thread, and always reconciles Runtime
605-
cleanup before completing or cancelling. Reports show 0–100 display scores,
603+
deploys a temporary Runtime, checkpoints per-case execution as JSONL, and
604+
judges batches in one fresh resumable Codex thread. Analysis and judging both
605+
deliver their contract through an app-server dynamic tool instead of parsing
606+
free text, and the analysis turn runs on a Studio background worker so a long
607+
analysis never waits inside the upload request. Each judged batch is a durable
608+
request the runner writes into the Session: Studio answers it with one
609+
app-server turn carrying the verdict, and the runner validates and caches that
610+
verdict before the next batch. A request stays on disk until it is answered, so
611+
a Studio restart replays the same batch; a request that cannot be answered
612+
falls back to `codex exec` inside the same batch budget. The request declares
613+
how long the runner will listen, and the turn is sized to answer inside that
614+
window, so even a judge batch that is too slow comes back as an answered
615+
failure rather than a timeout. Setting
616+
`AGENTKIT_MIGRATION_JUDGE_APP_SERVER=0` pins that scripted judge for the
617+
whole run instead of writing judge requests at all. Cleanup is always
618+
reconciled before completing or cancelling. Reports show 0–100 display scores,
606619
execution success, evidence coverage, N/A counts, low-scoring and failed
607620
cases, versions, evidence severity, and cleanup status without a pass/fail
608621
verdict. Each raw judge score is rounded half up to a 0–100 integer before

0 commit comments

Comments
 (0)