Skip to content

feat: verification probes with CI-aware execution policy - #5

Merged
eliahilse merged 3 commits into
mainfrom
feat/exec-probes
Jul 30, 2026
Merged

feat: verification probes with CI-aware execution policy#5
eliahilse merged 3 commits into
mainfrom
feat/exec-probes

Conversation

@eliahilse

Copy link
Copy Markdown
Owner

Reviewers can now execute, without duplicating CI.

  • Probes allowed: engines may run small targeted checks — bun -e / node -e snippets, exercising a single function against an edge case, git inspection — to verify a suspicion before reporting it.
  • CI-covered commands off-limits: kyora-review parses .github/workflows/* for run: commands, classifies suites/builds/linters/type-checks, and injects them into every engine prompt as banned ("if a finding depends on their outcome, state the expectation instead"). Re-running what the runner already runs is pure compute waste.
  • Enforcement where the CLI supports it: claude/kimi get Bash allowed with deny rules (test runners, bun run/npm run scripts, tsc, turbo, installs, rm/mv, git push/commit/checkout/reset); codex keeps its read-only sandbox which already permits execution without writes; grok/qwen get the policy via prompt.
  • 3 new unit tests for workflow parsing/classification (19 total).

Also restores the TOC README from #2 — it was stranded on feat/review-framework because #1 squash-merged to main before #2's merge landed on the branch, so main still had the old short README.

Includes #4 (branched from it) — merge #4 first and this diff collapses to the execution-policy changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CQy3ZJ5MyUxo4qjZwA93Na

eliahilse and others added 3 commits July 30, 2026 17:24
Both were resolving through hoisting locally but fail on the runner's
bun linker. CI now runs the same bun as development.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQy3ZJ5MyUxo4qjZwA93Na
Engines can now execute targeted probes (snippet evaluation, single-
function checks) to verify findings, while the commands CI already runs
are auto-detected from .github/workflows and declared off-limits —
re-running suites the runner executes anyway is pure compute waste.
Claude/kimi get Bash with deny rules for suites, builds, installs,
writes, and history-mutating git; codex keeps its read-only sandbox.

Also restores the TOC README from #2, which was stranded on
feat/review-framework because #1 squash-merged before #2 landed, and
ignores generated app artifacts at the root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQy3ZJ5MyUxo4qjZwA93Na
@eliahilse
eliahilse merged commit 6c20f1c into main Jul 30, 2026
2 checks passed
@eliahilse

Copy link
Copy Markdown
Owner Author

Added a sixth engine while this was open: glm — GLM-5.2 on the Z.ai Coding Plan. Driven through Claude Code against Z.ai's Anthropic-compatible endpoint rather than opencode: opencode's Z.ai provider currently deadlocks on GLM's streamed tool-call deltas (anomalyco/opencode#6708 / #8428 — reproduced locally, hangs on any prompt that triggers tool use). Validated end-to-end against the live endpoint with a planted-bug test. Key resolves from ZAI_API_KEY or opencode's auth store, so an existing opencode auth login just works.

@eliahilse

Copy link
Copy Markdown
Owner Author

Seventh engine slot filled while open: qwen now runs qwen3.8-max-preview (Alibaba Token Plan) through Claude Code against the Token Plan's Anthropic-compatible endpoint — validated live with the planted-bug test (found it, critical, correct NaN analysis). Replaces the unvalidated qwen-code CLI adapter. Key resolves from QWEN_API_KEY or the opencode config that bl config agent writes. opencode itself is benched as a harness for now: fresh-state runs wedge on any tool-triggering prompt across three different providers (Z.ai, opencode-go, bailian) on this machine.

eliahilse added a commit that referenced this pull request Jul 30, 2026
Two things:

**Action no longer needs npm.** `@kyora-sh/review` has zero runtime
dependencies, so the action now runs the CLI source straight from its
own checkout (`version: bundled`, the new default) — `uses:
eliahilse/kyora/action@<ref>` pins the code version naturally, and
publishing to npm becomes optional (still available via `version:
<npm-version>` for anyone who wants it). This unblocks CI installs
entirely: workflow file + secrets, nothing else.

**Recovers the glm + qwen engines.** The #5 squash-merge landed before
the last two engine commits reached the branch (same stranding as the
README in #2), so main was missing them. Cherry-picked: `glm` (GLM-5.2,
Z.ai coding plan via Claude Code) and `qwen` (qwen3.8-max-preview, Token
Plan via Claude Code) — both live-validated earlier. Full panel on main
after this: codex, claude, kimi, glm, grok, qwen.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01CQy3ZJ5MyUxo4qjZwA93Na

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant