Token Optimizer ranks zero-spend runs: top-workflows aggregation should exclude runs that consumed 0 AIC (skipped via activation)

[mrfelton]

Summary

In workflows/agentic-token-optimizer.md, the Aggregate top workflows by AIC usage step keeps every completed run, including runs where the agent never executed (activation/preconditions caused a skip) and therefore consumed 0 AIC. These zero-spend runs pollute the candidate ranking that drives target selection.

This is a distinct issue from the self-targeting guard in #119, though they compound each other.

The bug

Aggregate step (line ~71):

[.runs[]
  | select(.status == "completed")
  | { workflow_name, ai_credits: (.aic // 0), ... }
]
| group_by(.workflow_name)
| map({ ..., run_count: length, total_ai_credits: ..., avg_ai_credits: ((map(.ai_credits) | add) / length), ... })
| sort_by(.total_ai_credits) | reverse | .[:10]

A run that skipped its agent still has status == "completed" but aic == 0. Because the filter only checks status, those runs are included, which:

• pads run_count for workflows that mostly skip,
• dilutes avg_ai_credits (the 0s drag the average down), and
• can surface workflows into the top-10 that never actually spent any credits in the window.

High-frequency, event-driven workflows (e.g. triggered on issue/PR labels) are exactly this case: they run very often but skip the agent on most invocations, so they contribute a large number of 0-AIC completed runs.

Suggested fix

Exclude runs that consumed nothing from the spend ranking, right after the status filter:

 [.runs[]
   | select(.status == "completed")
+  | select((.aic // 0) > 0)
   | {
       workflow_name: .workflow_name,
       ai_credits: (.aic // 0),
       ...
     }
 ]

A "top workflows by AIC spend" ranking should only count runs that actually spent AIC.

⚠️ Compiler-version dependency (important)

This fix is only safe once the compiler reliably computes AIC. On older compilers — including v0.72.1, which this repo currently pins (.github/aw/actions-lock.json, lock compiler_version) — the Copilot-engine 0-AIC bug is present: provider github_models was not normalized to github-copilot, so genuinely-expensive Copilot runs can report aic == 0. (Fixed in gh-aw v0.79.4 via github/gh-aw#38314, with related span fixes in #38364 and #38327/#38353/#38412.)

On v0.72.1, select((.aic // 0) > 0) would therefore drop real-spend runs that were merely miscalculated as 0 — making the ranking less accurate. So this change should be paired with / gated on bumping the gh-aw compiler to ≥ v0.79.4 for these workflows. Once on ≥ v0.79.4, aic == 0 reliably means "agent skipped / no consumption", and the filter is correct.

Context

• We run these workflows in production in a large monorepo. On gh-aw v0.79.4 this filter (together with the Token Optimizer self-exclusion never matches: guard checks for "Token" in a name that is "AIC Usage Optimizer/Audit" #119 self-exclusion fix) makes the optimizer correctly target genuinely-expensive workflows instead of high-frequency skip-heavy ones.
• Source at githubnext/agentic-ops@c31cadc6b436d75014a8e2ad34dcfd4c5c079256; bug present on current default branch.
• Related: Token Optimizer self-exclusion never matches: guard checks for "Token" in a name that is "AIC Usage Optimizer/Audit" #119 (self-exclusion guard never matches).