feat(harness): judge the advertised skills and the transfer target

Two more places decide with nothing but wording. The skills callback
advertises every loaded skill in the agent instruction, so a large library
spends prompt budget on skills the request will never use, and the model picks
from a list nobody narrowed. Routing a request to a sub-agent has the same
shape: the model reads the agent descriptions and calls ``transfer_to_agent``
on its own.

``HarnessSkillPrefilterPlugin`` asks one ``noul`` question per advertised skill
in a single request and rewrites the skill list of that request only. The
candidates are judged independently instead of as one ``choice``, because a
request often needs two skills and a choice names one winner, and the
descriptions travel in the questions so one skill's description cannot decide
another skill's answer. The agent instruction keeps every skill, so the next
request starts from the full list, and a skill the judgement did not answer for
stays advertised: a missing probability is not evidence of irrelevance.

``HarnessAgentRoutingPlugin`` asks one ``choice`` question whose options are
the transfer targets and whose option descriptions are theirs, and returns the
same ``transfer_to_agent`` call the model would have produced once the
judgement clears ``HARNESS_ROUTING_DECISION_THRESHOLD``. Everything less
certain stays with the model, which still sees the full transfer instructions,
and a judgement never names a target the transfer tool does not expose, so ADK
cannot be asked to resolve an agent it does not have.

Both are opt-in components (``skill_prefilter`` and ``agent_routing``), judge at
most once per invocation, judge only the text of the user's message, and fall
back to their rule when no decision model is configured. The READMEs and the
environment-variable reference document the five new settings.

Change-Id: Ibd735d56a515b50687d48d3592859323d73a6a11

Author: wujiaming-ai

docs/content/docs/references/configuration/environment-variables.en.mdx

@@ -84,6 +84,11 @@ Prefix `HARNESS_`, used to attach optional Harness plugins to HarnessApp Runtime
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | Long-run steering: guidance is injected when the judged probability of being ready is at or above this value; default `0.5`. |
 | `HARNESS_MODE_DECISION_THRESHOLD` | Context mode blocks: a block is injected when the judged probability is at or above this value; default `0.5`. |
 | `HARNESS_VERIFIER_SUPPORT_THRESHOLD` | Final-answer support: the answer fails when the judged support is below this value; default `0.5`. |
+| `HARNESS_SKILL_STRATEGY` | Advertised-skills strategy, `all` or `decision`; default `all`. Requires the `skill_prefilter` component. |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | Advertised skills: a skill stays in the request when the judged probability of needing it is at or above this value; default `0.5`. |
+| `HARNESS_SKILL_MAX_CANDIDATES` | Advertised skills: a list longer than this is not judged, so every skill stays advertised; default `40`. |
+| `HARNESS_ROUTING_STRATEGY` | Sub-agent routing strategy, `model` or `decision`; default `model`. Requires the `agent_routing` component. |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | Routing: the judged agent is transferred to only at or above this probability; default `0.5`. |
 
 The `harness_enhance` block maps to these environment variables when deploying a HarnessApp Runtime. Prefer `harness.yaml` or `veadk agentkit invoke` flags for normal developer workflows; use environment variables for platform integration and container runtimes.
 

docs/content/docs/references/configuration/environment-variables.mdx

@@ -84,6 +84,11 @@ volcengine:
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | 长任务收尾阈值，默认 `0.5`；判定可收尾概率高于该值即注入引导。 |
 | `HARNESS_MODE_DECISION_THRESHOLD` | 上下文模式块阈值，默认 `0.5`；判定概率高于该值即注入对应模式块。 |
 | `HARNESS_VERIFIER_SUPPORT_THRESHOLD` | 最终回答支撑度阈值，默认 `0.5`；判定支撑度低于该值即判为失败。 |
+| `HARNESS_SKILL_STRATEGY` | 技能广告策略，`all` 或 `decision`，默认 `all`；需要 `skill_prefilter` 组件。 |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | 技能广告阈值，默认 `0.5`；判定需要该技能的概率不低于该值才继续广告。 |
+| `HARNESS_SKILL_MAX_CANDIDATES` | 技能候选上限，默认 `40`；技能数量超过该值时不判定，全部照常广告。 |
+| `HARNESS_ROUTING_STRATEGY` | 子 Agent 路由策略，`model` 或 `decision`，默认 `model`；需要 `agent_routing` 组件。 |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | 路由阈值，默认 `0.5`；判定概率不低于该值才直接转移。 |
 
 `harness_enhance` 配置块会在 HarnessApp Runtime 部署时映射为这些环境变量。推荐开发者优先通过 `harness.yaml` 或 `veadk agentkit invoke` 参数启用，环境变量适合平台集成和镜像运行时。
 

docs/extensions/harness/README.md

@@ -60,6 +60,8 @@ plugins = build_harness_plugins(components=["compactor"])
 | `compactor` | `HarnessCompressPlugin` | Compacts oversized tool results and old function responses. |
 | `response_verification` | `HarnessResponseVerificationPlugin` | Records tool receipts and checks whether final answers are supported. |
 | `long_run_control` | `HarnessLongRunControlPlugin` | Adds finish-oriented guidance when a run approaches its model-call budget. |
+| `skill_prefilter` | `HarnessSkillPrefilterPlugin` | Advertises only the skills the current request needs; the agent instruction keeps every skill. |
+| `agent_routing` | `HarnessAgentRoutingPlugin` | Transfers to the sub-agent a confident judgement picked; the model routes everything else. |
 
 ## Core Concepts
 
@@ -88,6 +90,10 @@ plugins = build_harness_plugins(components=["compactor"])
 | `veadk/extensions/harness/plugins/compactor/` | Tool-result and context compaction callback plugin. |
 | `veadk/extensions/harness/plugins/response_verification/` | Receipt recording and final-response verification callback plugin. |
 | `veadk/extensions/harness/plugins/long_run_control/` | Long-run guidance callback plugin. |
+| `veadk/extensions/harness/modules/skill_prefilter/` | Skill-list parsing and the per-skill judgement. |
+| `veadk/extensions/harness/modules/agent_routing/` | Transfer-target judgement. |
+| `veadk/extensions/harness/plugins/skill_prefilter/` | Skill-list narrowing callback plugin. |
+| `veadk/extensions/harness/plugins/agent_routing/` | Transfer callback plugin. |
 | `veadk/extensions/harness/plugins/_shared/` | Internal callback helpers shared by plugins. |
 | `veadk/extensions/harness/stores/` | Store protocol and in-memory or JSONL implementations. |
 
@@ -177,6 +183,8 @@ export HARNESS_VERIFIER_MODE=observe
 # export HARNESS_COMPACTION_STRATEGY=decision
 # export HARNESS_LONG_RUN_STRATEGY=decision
 # export HARNESS_MODE_STRATEGY=decision
+# export HARNESS_SKILL_STRATEGY=decision
+# export HARNESS_ROUTING_STRATEGY=decision
 ```
 
 Equivalent YAML:
@@ -220,10 +228,15 @@ veadk agentkit invoke \
 | `HARNESS_COMPACTION_KEEP_THRESHOLD` | `0.5` | Compaction candidates: keeps a candidate above this probability. |
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | `0.5` | Long-run steering: steers the run to finish above this probability. |
 | `HARNESS_MODE_DECISION_THRESHOLD` | `0.5` | Context mode blocks: injects a block above this probability. |
+| `HARNESS_SKILL_STRATEGY` | `all` | Advertised skills: `all` or `decision`. |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | `0.5` | Skills: a skill stays advertised when its judged probability is at or above this value. |
+| `HARNESS_SKILL_MAX_CANDIDATES` | `40` | Skills: a list longer than this is not judged at all, so every skill stays advertised. |
+| `HARNESS_ROUTING_STRATEGY` | `model` | Agent routing: `model` or `decision`. |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | `0.5` | Routing: transfers only when the judged agent is at or above this probability. |
 
 ## Decision Model Strategies
 
-The four `*_STRATEGY=decision` settings replace a rule with a judgement from
+The six `*_STRATEGY=decision` settings replace a rule with a judgement from
 the configured decision model. They need `DECISION_MODEL_ENABLED=true` and an
 API key; without one, each strategy keeps its rule and logs a warning.
 
@@ -233,6 +246,8 @@ API key; without one, each strategy keeps its rule and logs a warning.
 | `HARNESS_LONG_RUN_STRATEGY` | Model-call counter | Counter, forced after the unconditional count |
 | `HARNESS_MODE_STRATEGY` | Precision and artifact keyword markers | Keyword markers |
 | `HARNESS_VERIFIER_STRATEGY` | Completion markers plus a successful-receipt check | Builtin rules |
+| `HARNESS_SKILL_STRATEGY` | Advertising every loaded skill | Every skill stays advertised |
+| `HARNESS_ROUTING_STRATEGY` | The model picking the sub-agent to transfer to | The model routes |
 
 ### Judgement Thresholds
 
@@ -248,6 +263,8 @@ back to `0.5` for an unusable one.
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | `0.5` | Steers a run toward its answer sooner |
 | `HARNESS_MODE_DECISION_THRESHOLD` | `0.5` | Injects the mode block more often |
 | `HARNESS_VERIFIER_SUPPORT_THRESHOLD` | `0.5` | Requires more evidence before the answer passes |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | `0.5` | Hides more skills from the list |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | `0.5` | Routes more requests without asking the model |
 
 A judgement also picks an action: long-run steering chooses `narrow_scope` /
 `nudge_to_finish` / `force_finish` to shape the injected guidance, and

docs/extensions/harness/README.zh.md

@@ -55,6 +55,8 @@ plugins = build_harness_plugins(components=["compactor"])
 | `compactor` | `HarnessCompressPlugin` | 压缩过大的工具结果和旧 function response。 |
 | `response_verification` | `HarnessResponseVerificationPlugin` | 记录 tool receipt，并检查最终回答是否有证据支撑。 |
 | `long_run_control` | `HarnessLongRunControlPlugin` | 当运行接近模型调用预算时，注入面向收敛的引导。 |
+| `skill_prefilter` | `HarnessSkillPrefilterPlugin` | 每次请求只广告本次需要的技能；agent 指令本身保留完整列表。 |
+| `agent_routing` | `HarnessAgentRoutingPlugin` | 判定足够确信时直接转给对应子 Agent，其余请求仍由对话模型路由。 |
 
 ## 核心概念
 
@@ -83,6 +85,10 @@ plugins = build_harness_plugins(components=["compactor"])
 | `veadk/extensions/harness/plugins/compactor/` | 工具结果和上下文压缩回调 plugin。 |
 | `veadk/extensions/harness/plugins/response_verification/` | Receipt 记录和最终回答校验回调 plugin。 |
 | `veadk/extensions/harness/plugins/long_run_control/` | 长任务收敛引导回调 plugin。 |
+| `veadk/extensions/harness/modules/skill_prefilter/` | 技能列表解析与逐候选判定。 |
+| `veadk/extensions/harness/modules/agent_routing/` | 转移目标判定。 |
+| `veadk/extensions/harness/plugins/skill_prefilter/` | 技能列表收窄回调 plugin。 |
+| `veadk/extensions/harness/plugins/agent_routing/` | 转移回调 plugin。 |
 | `veadk/extensions/harness/plugins/_shared/` | 多个 plugin 共享的内部回调工具。 |
 | `veadk/extensions/harness/stores/` | Store 协议，以及内存 / JSONL 实现。 |
 
@@ -169,6 +175,8 @@ export HARNESS_VERIFIER_MODE=observe
 # export HARNESS_COMPACTION_STRATEGY=decision
 # export HARNESS_LONG_RUN_STRATEGY=decision
 # export HARNESS_MODE_STRATEGY=decision
+# export HARNESS_SKILL_STRATEGY=decision
+# export HARNESS_ROUTING_STRATEGY=decision
 ```
 
 等价 YAML：
@@ -212,17 +220,24 @@ veadk agentkit invoke \
 | `HARNESS_COMPACTION_KEEP_THRESHOLD` | `0.5` | 压缩候选：概率高于该值即保留。 |
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | `0.5` | 长任务引导：概率高于该值即引导收尾。 |
 | `HARNESS_MODE_DECISION_THRESHOLD` | `0.5` | 上下文模式块：概率高于该值即注入。 |
+| `HARNESS_SKILL_STRATEGY` | `all` | 技能广告策略：`all` 或 `decision`。 |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | `0.5` | 技能：判定概率不低于该值才继续广告。 |
+| `HARNESS_SKILL_MAX_CANDIDATES` | `40` | 技能：列表超过该数量时不做判定，全部照常广告。 |
+| `HARNESS_ROUTING_STRATEGY` | `model` | 子 Agent 路由策略：`model` 或 `decision`。 |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | `0.5` | 路由：判定概率不低于该值才直接转移。 |
 
 ## 判定模型策略
 
-四个 `*_STRATEGY=decision` 开关把一条规则换成判定模型的判定结果，需要 `DECISION_MODEL_ENABLED=true` 与 API Key；没有配置时各自保留原规则并打印告警。
+六个 `*_STRATEGY=decision` 开关把一条规则换成判定模型的判定结果，需要 `DECISION_MODEL_ENABLED=true` 与 API Key；没有配置时各自保留原规则并打印告警。
 
 | 策略 | 被替代的规则 | 判定不可用时 |
 | --- | --- | --- |
 | `HARNESS_COMPACTION_STRATEGY` | 按角色和长度挑选压缩候选 | 内置规则 |
 | `HARNESS_LONG_RUN_STRATEGY` | 仅按模型调用次数计数 | 计数规则，超过强制次数后必定生效 |
 | `HARNESS_MODE_STRATEGY` | 精度/产物关键词匹配 | 关键词匹配 |
 | `HARNESS_VERIFIER_STRATEGY` | 完成类关键词加「有无成功回执」 | 内置规则 |
+| `HARNESS_SKILL_STRATEGY` | 广告全部已加载技能 | 技能列表保持不变 |
+| `HARNESS_ROUTING_STRATEGY` | 由对话模型选择要转移的子 Agent | 由对话模型路由 |
 
 ### 判定阈值
 
@@ -234,6 +249,8 @@ veadk agentkit invoke \
 | `HARNESS_LONG_RUN_READY_THRESHOLD` | `0.5` | 更早把运行推向收尾 |
 | `HARNESS_MODE_DECISION_THRESHOLD` | `0.5` | 更频繁注入模式块 |
 | `HARNESS_VERIFIER_SUPPORT_THRESHOLD` | `0.5` | 要求更充分的证据才放行回答 |
+| `HARNESS_SKILL_DECISION_THRESHOLD` | `0.5` | 从列表里隐藏更多技能 |
+| `HARNESS_ROUTING_DECISION_THRESHOLD` | `0.5` | 更多请求不经对话模型直接转移 |
 
 判定还会选动作：长任务引导可选 `narrow_scope` / `nudge_to_finish` / `force_finish` 决定注入的引导文案，最终回答校验可选 `retry_tool_call` / `soften_claim` / `drop_claim` / `ask_user` 决定修复指引；动作不可用时保留默认文案，评级仍然生效。
 

tests/extensions/decisions/test_harness_judges.py

@@ -30,10 +30,17 @@
     ToolResultCompactor,
     ToolResultCompactorConfig,
 )
+from veadk.extensions.harness.modules.agent_routing import DecisionAgentRouter
+from veadk.extensions.harness.modules.skill_prefilter import DecisionSkillJudge
 from veadk.extensions.harness.schemas import CompressionRequest, ConversationMessage
 
 from .fake_system_one import fake_system_one
 
+_ROUTING_AGENTS = {
+    "billing_agent": "handles invoices and refunds",
+    "docs_agent": "answers product questions",
+}
+
 
 def _extension(base_url: str) -> DecisionExtension:
     return DecisionExtension(
@@ -147,3 +154,97 @@ def test_decision_strategy_end_to_end_keeps_evidence_and_fits() -> None:
     assert result.report.compressed_chars <= 12000
     assert result.messages[1] == messages[1]
     assert result.messages[3] != messages[3]
+
+
+def _noul_script(
+    values: dict[str, float],
+) -> tuple[int, dict[str, str], dict[str, object]]:
+    """Build a response answering ``skill_<index>`` questions."""
+    return (
+        200,
+        {},
+        {
+            "model": "fake-system-one",
+            "answers": {
+                name: {"type": "noul", "noul": value} for name, value in values.items()
+            },
+        },
+    )
+
+
+def test_skill_judge_asks_about_every_candidate_in_one_request() -> None:
+    with fake_system_one([_noul_script({"skill_0": 0.92, "skill_1": 0.08})]) as server:
+        judge = DecisionSkillJudge(_extension(server.base_url))
+        probabilities = asyncio.run(
+            judge.aprobabilities(
+                user_input="render the chart",
+                skills={"chart_skill": "draws charts", "mail_skill": "sends mail"},
+            )
+        )
+
+    assert len(server.calls) == 1
+    call = server.calls[0]
+    assert sorted(call.questions) == ["skill_0", "skill_1"]
+    assert call.questions["skill_1"]["type"] == "noul"
+    assert probabilities["chart_skill"] == pytest.approx(0.92)
+    assert probabilities["mail_skill"] == pytest.approx(0.08)
+    # 候选只出现在问题里：状态没有技能描述，问题之间互相看不见
+    assert "render the chart" in call.state
+    assert "draws charts" not in call.state
+    assert "draws charts" in call.questions["skill_0"]["instructions"]
+
+
+def test_agent_router_returns_a_target_above_its_threshold() -> None:
+    scripted = (
+        200,
+        {},
+        {
+            "model": "fake-system-one",
+            "answers": {
+                "target": {
+                    "type": "choice",
+                    "choice": "docs_agent",
+                    "confidence": 0.83,
+                    "probabilities": {"docs_agent": 0.83, "billing_agent": 0.1},
+                }
+            },
+        },
+    )
+    with fake_system_one([scripted]) as server:
+        router = DecisionAgentRouter(
+            _extension(server.base_url), confidence_threshold=0.8
+        )
+        target = asyncio.run(
+            router.aroute(user_input="how do I rotate a key?", agents=_ROUTING_AGENTS)
+        )
+
+    assert target == "docs_agent"
+    assert len(server.calls) == 1
+    call = server.calls[0]
+    assert call.questions["target"]["criteria"] == _ROUTING_AGENTS
+    assert "how do I rotate a key?" in call.state
+
+
+def test_agent_router_leaves_a_low_confidence_choice_to_the_model() -> None:
+    scripted = (
+        200,
+        {},
+        {
+            "model": "fake-system-one",
+            "answers": {
+                "target": {
+                    "type": "choice",
+                    "choice": "docs_agent",
+                    "confidence": 0.55,
+                    "probabilities": {"docs_agent": 0.55, "billing_agent": 0.45},
+                }
+            },
+        },
+    )
+    with fake_system_one([scripted]) as server:
+        router = DecisionAgentRouter(
+            _extension(server.base_url), confidence_threshold=0.8
+        )
+        target = asyncio.run(router.aroute(user_input="hello", agents=_ROUTING_AGENTS))
+
+    assert target is None