fix: react on the default token, and let the recipe pick the judge model (#42)

Two things, both measured on a real run rather than reasoned about.

REACTIONS NEVER HAPPENED. The gate read `gh api user --jq .type` and declined
unless it saw "Bot". That endpoint needs a user-scoped token, and github-token
defaults to ${{ github.token }} — an installation token, for which it returns no
user. The `|| echo "Bot"` fallback was meant to cover that and did not: the call
can exit 0 with no `.type`, so ME_TYPE came back empty, `!= "Bot"` held, and both
the 👀 and the settle step skipped — for every consumer on the default token,
which is the case the feature exists to serve. A run logged "github-token
belongs to a user, not an app" while the reaction already on the PR was authored
by github-actions[bot].

Inverted to fail towards acting: a PAT identifies itself as type User and
declines; 403, empty and Bot all react. That also unsticks the 👀 left behind by
runs that predate the settle step, since the clear now runs.

THE JUDGE MODEL COMES FROM THE RECIPE. judge-model defaulted to a concrete
model, so the L2 call named it directly and never reached the router — the one
routing decision in the product that a workspace could not make for itself. Its
default is now empty, which sends the router alias, and judge.mjs stamps
`x-cr-lens: judge` so the recipe's judge rule chooses.

No model change: the shipped recipe routes that rule to deepseek-v4-pro, which
is what judge-model used to name. Setting judge-model explicitly still pins a
model outright, regardless of the recipe.

The header goes out unconditionally, including when --model names a concrete
model: nothing resolves an alias then, so nothing reads it, and "is this an
alias" is the gateway's judgement rather than this script's.

ORDER: OrcaRouter-O2#1433 puts the judge rule in the provisioned recipe and has
to be deployed first. Without it the judge lands on the recipe's default — the
model it is scoring — which agrees with itself while still reporting success.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: ZhenghuaBao

action.yml

@@ -157,7 +157,7 @@ inputs:
       model so the judge acts as an independent second opinion. Ignored when
       `precision-filter` is `"false"`.
     required: false
-    default: "deepseek/deepseek-v4-pro"
+    default: ""
   judge-threshold:
     description: >-
       Keep-threshold for the L2 judge's per-cluster confidence score (0–1).
@@ -286,8 +286,19 @@ runs:
         #
         # Checked once here rather than inside react(), so a token that cannot be
         # identified costs one API call, not two.
-        ME_TYPE=$(gh api user --jq .type 2>/dev/null || echo "Bot")
-        if [ "$ME_TYPE" != "Bot" ]; then
+        # ONLY AN EXPLICIT "User" DECLINES. `gh api user` needs a user-scoped
+        # token, and github-token defaults to ${{ github.token }} — an
+        # installation token, for which that endpoint returns no user. The old
+        # `|| echo "Bot"` fallback was meant to cover that and did not: the call
+        # can exit 0 with no `.type`, so ME_TYPE came back empty, `!= "Bot"` held,
+        # and the reaction was skipped for EVERY consumer on the default token —
+        # the case this exists to serve. Measured on a real run: "belongs to a
+        # user, not an app" while the reaction author was github-actions[bot].
+        #
+        # Inverted to fail towards acting: a PAT identifies itself as type User,
+        # and anything else — 403, empty, Bot — reacts.
+        ME_TYPE=$(gh api user --jq .type 2>/dev/null) || ME_TYPE=""
+        if [ "$ME_TYPE" = "User" ]; then
           echo "github-token belongs to a user, not an app; not reacting"
           exit 0
         fi
@@ -921,11 +932,25 @@ runs:
             POLICY_BLOCK_SNAPSHOT="${POLICY_BLOCK}.pre-l2"
             rm -f "$POLICY_BLOCK_SNAPSHOT"
             if [ -f "$POLICY_BLOCK" ]; then cp -f "$POLICY_BLOCK" "$POLICY_BLOCK_SNAPSHOT"; fi
-            if node "$JUDGE" "$1" --model "$JUDGE_MODEL" --threshold "$JUDGE_THRESHOLD" --out "$L2_OUT" 2>&1 \
+            # THE ROUTER DECIDES, unless judge-model overrides it. Its default is now
+            # empty, which sends this router's ALIAS instead of a model name; judge.mjs
+            # stamps x-cr-lens: judge, so the recipe's judge rule picks the model.
+            #
+            # Same model as before — the shipped recipe routes that rule to
+            # deepseek-v4-pro, which is what judge-model used to name. What moves is
+            # where the choice lives, so a workspace can price its judge by editing its
+            # own recipe rather than waiting for a release.
+            #
+            # A recipe with NO judge rule sends the judge to its default, i.e. the
+            # reviewer's own model, which agrees with itself while still reporting
+            # success. That is why the provisioned recipe carries the rule, and why
+            # setting judge-model explicitly stays the way to pin a model outright.
+            L2_MODEL="${JUDGE_MODEL:-$ROUTER}"
+            if node "$JUDGE" "$1" --model "$L2_MODEL" --threshold "$JUDGE_THRESHOLD" --out "$L2_OUT" 2>&1 \
                && [ -s "$L2_OUT" ]; then
               L2_IN_COUNT=$(node -pe "(require('$1').comments||[]).length")
               L2_OUT_COUNT=$(node -pe "(require('$L2_OUT').comments||[]).length")
-              echo "L2 judge ($JUDGE_MODEL, thr=$JUDGE_THRESHOLD): $L2_IN_COUNT -> $L2_OUT_COUNT"
+              echo "L2 judge ($L2_MODEL, thr=$JUDGE_THRESHOLD): $L2_IN_COUNT -> $L2_OUT_COUNT"
               mv "$L2_OUT" "$1"
               # L2 succeeded. Restore engine's block if we snapshotted one —
               # covers the case where L2 was warn-mode guardrail-hit and
@@ -1580,8 +1605,19 @@ runs:
         # of answering it more precisely each time.
         #
         # The review body and the summary say everything the reactions would.
-        ME_TYPE=$(gh api user --jq .type 2>/dev/null || echo "Bot")
-        if [ "$ME_TYPE" != "Bot" ]; then
+        # ONLY AN EXPLICIT "User" DECLINES. `gh api user` needs a user-scoped
+        # token, and github-token defaults to ${{ github.token }} — an
+        # installation token, for which that endpoint returns no user. The old
+        # `|| echo "Bot"` fallback was meant to cover that and did not: the call
+        # can exit 0 with no `.type`, so ME_TYPE came back empty, `!= "Bot"` held,
+        # and the reaction was skipped for EVERY consumer on the default token —
+        # the case this exists to serve. Measured on a real run: "belongs to a
+        # user, not an app" while the reaction author was github-actions[bot].
+        #
+        # Inverted to fail towards acting: a PAT identifies itself as type User,
+        # and anything else — 403, empty, Bot — reacts.
+        ME_TYPE=$(gh api user --jq .type 2>/dev/null) || ME_TYPE=""
+        if [ "$ME_TYPE" = "User" ]; then
           echo "github-token belongs to a user, not an app; leaving reactions alone"
           exit 0
         fi

scripts/judge.mjs

@@ -6,7 +6,11 @@
 // correct, high-value defect in THIS change, (3) recommends keep/drop. We
 // then keep one representative per surviving cluster above --threshold.
 //
-//   node judge.mjs <filtered.json> [--out f] [--threshold 0.7] [--model deepseek/deepseek-v4-pro]
+//   node judge.mjs <filtered.json> [--out f] [--threshold 0.7] [--model orcarouter/<router>]
+//
+// --model takes either a router ALIAS (the normal case — the workspace's recipe
+// then picks the model, keyed on the `x-cr-lens: judge` header this sends) or a
+// concrete model name, which pins it regardless of the recipe.
 //
 // LLM connection resolution:
 //   1. OCR_LLM_URL / OCR_LLM_TOKEN / OCR_LLM_AUTH_HEADER env vars (production
@@ -117,7 +121,23 @@ const body = JSON.stringify({
 
 const res = await fetch(llmUrl, {
   method: "POST",
-  headers: { "content-type": "application/json", [llmAuthHeader]: "Bearer " + llmToken },
+  headers: {
+    "content-type": "application/json",
+    [llmAuthHeader]: "Bearer " + llmToken,
+    // THE ANGLE, so a router recipe can put this call on its own model.
+    //
+    // --model is normally the router ALIAS (action.yml passes it when judge-model
+    // is unset), and an alias resolves through the workspace's DSL, which has no
+    // other way to tell a judge call from a review call: the reviewer stamps no
+    // angle. Without this header the judge takes the recipe's default — the
+    // reviewer's own model — and a judge scoring work its own model produced
+    // agrees with it, so the pass goes inert while still reporting success.
+    //
+    // Harmless when --model names a concrete model: nothing resolves an alias, so
+    // nothing reads the header. Sent unconditionally rather than only for aliases
+    // because "is this an alias" is the gateway's judgement, not this script's.
+    "x-cr-lens": "judge",
+  },
   body,
 });
 const raw = await res.text();

scripts/judge.test.mjs

@@ -454,3 +454,12 @@ describe("request shape", () => {
     assert.equal(r.read().model, "engine-model");
   });
 });
+
+// The judge's model normally comes from the workspace recipe, and the recipe has no
+// other way to tell this call from a review call — the reviewer stamps no angle. A
+// missing header sends the judge to the recipe's default, i.e. the model it is
+// scoring, which agrees with itself while still reporting success.
+test("the request stamps x-cr-lens: judge so a recipe can route it", () => {
+  const src = readFileSync(new URL("./judge.mjs", import.meta.url), "utf8");
+  assert.match(src, /"x-cr-lens":\s*"judge"/, "the judge call must declare its angle");
+});