@@ -600,9 +600,22 @@ See [deployment and operation](service/studio_release_notifier/README.md).
600600 the framework, entry point, and open questions. Structured frameworks run the
601601 preinstalled ` ak migrate ` ; Dify and Any projects run
602602 ` ak migrate --execution in-place ` with Codex in the same Session. Evaluation
603- deploys a temporary Runtime, checkpoints per-case execution as JSONL, judges
604- batches in one fresh resumable Codex thread, and always reconciles Runtime
605- cleanup before completing or cancelling. Reports show 0–100 display scores,
603+ deploys a temporary Runtime, checkpoints per-case execution as JSONL, and
604+ judges batches in one fresh resumable Codex thread. Analysis and judging both
605+ deliver their contract through an app-server dynamic tool instead of parsing
606+ free text, and the analysis turn runs on a Studio background worker so a long
607+ analysis never waits inside the upload request. Each judged batch is a durable
608+ request the runner writes into the Session: Studio answers it with one
609+ app-server turn carrying the verdict, and the runner validates and caches that
610+ verdict before the next batch. A request stays on disk until it is answered, so
611+ a Studio restart replays the same batch; a request that cannot be answered
612+ falls back to ` codex exec ` inside the same batch budget. The request declares
613+ how long the runner will listen, and the turn is sized to answer inside that
614+ window, so even a judge batch that is too slow comes back as an answered
615+ failure rather than a timeout. Setting
616+ ` AGENTKIT_MIGRATION_JUDGE_APP_SERVER=0 ` pins that scripted judge for the
617+ whole run instead of writing judge requests at all. Cleanup is always
618+ reconciled before completing or cancelling. Reports show 0–100 display scores,
606619 execution success, evidence coverage, N/A counts, low-scoring and failed
607620 cases, versions, evidence severity, and cleanup status without a pass/fail
608621 verdict. Each raw judge score is rounded half up to a 0–100 integer before
0 commit comments