Skip to content

host-grok: Grok Build CLI as the 11th implemented harness host (proposal for ADR-278) #279

Description

@HF-teamdev

Verified against grok 1.0.17 (a549186d9d39) [alpha] (~/.grok/bin/grok, macOS arm64) on 2026-09-03. Repo paths cite ruvnet/metaharness at commit 42f568b7297c59065ea562247937bb25cd353a6a (feat(field-memory): add governed packed attractor routing (#202), committed 2026-09-02 02:31 -0400; shallow sparse-checkout --filter=blob:none clone made 2026-09-03).

TL;DR

Grok Build CLI already reads most of a Claude-shaped scaffold through its default-on [compat.claude] layer, and it has a project-scoped native surface of its own: .grok/config.toml for [mcp_servers] and [permission], .grok/hooks/*.json, .grok/agents/*.md, .grok/skills/, and AGENTS.md. host-grok is therefore the instruction and agent half of host-claude-code plus a small native .grok/ overlay, with strict renderers for the three things compat does not deliver: MCP servers, permission rules, and hooks.

One gate dominates the design. Project MCP servers, project hooks and project permission rules, in any format, are inert until the folder is trusted; grok inspect lists the MCP server anyway; and a fresh scaffold is untrusted by definition. The adapter has to be fail-closed about that (section 4.2), and the real-install check has to lift the gate explicitly (section 5).

This complements #160, which asks for Grok as a pipeline-stage model. This issue is about what npx metaharness my-bot --host grok writes. The two touch disjoint files; section 1 proposes who owns the package name.

Legend. A bare path:line is a repo path at 42f568b; NN-file.md:line is the user guide shipped inside grok 1.0.17 (extracted to ~/.grok/docs/user-guide/). VERIFIED means observed on the installed binary on 2026-09-03 (a command output or a file Grok wrote); DOC-ONLY means the guide asserts it and I did not observe it; outside the contract table in section 3, every statement is DOC-ONLY unless tagged. Things I never exercised use the not attempted: <reason> form (docs/dream-cycle/PROMPT.md:746-747). Nothing under ~/.grok, ~/.claude or the clone was modified; no credential values were read.

1. Existing issue audit

  • Feature request: Support for GLM and Grok #160 (open, 2026-08-02, "Feature request: Support for GLM and Grok") asks for @metaharness/host-grok as a second-opinion critic stage driven by grok --sandbox read-only --reasoning-effort <level> -p. Its flags exist on 1.0.17: --reasoning-effort with alias --effort (grok --help lines 110-113) and --sandbox (lines 124-127). Two things to settle. (a) Feature request: Support for GLM and Grok #160's premise that "metaharness currently ships host-claude-code and host-codex" is stale; HOSTS lists ten at packages/create-agent-harness/src/index.ts:70. (b) Feature request: Support for GLM and Grok #160 and this issue ask for the same package name. Proposal: @metaharness/host-grok is this issue's config-emission adapter, consistent with every host-* package ("The adapter remains config emission only", docs/adrs/ADR-247-host-prime-agent.md:58; "The adapter is a config-emitter, not a runtime shim", docs/adrs/ADR-036-host-opencode.md:31; HostAdapter is { name, generateConfig }, packages/kernel-js/src/types.ts:53-57). Feature request: Support for GLM and Grok #160's headless critic stage is a router or model-tier feature (@metaharness/router, README.md:88, :163-165; the cascade table at README.md:414-417) and should not be a host-* package. If you prefer otherwise, name the package Feature request: Support for GLM and Grok #160 should use and I will retitle. Either way, a shipped host-grok gives Feature request: Support for GLM and Grok #160's stage a committed [permission] posture to run under.
  • Codex host drops declared lifecycle hooks; add native hooks.json generation and reconcile stale host docs #168 (open, 2026-08-04) on Codex hooks: its three findings carry over to Grok. The matcher is a regex, not a command predicate (10-hooks.md:163); the nested hookSpecificOutput.permissionDecision form is accepted and wins when present (10-hooks.md:269), so one handler shape serves Claude Code, Codex and Grok; project hooks are trust-gated (10-hooks.md:79). Deltas: 15 events with no PermissionRequest (10-hooks.md:89-105), handler types command and http only (10-hooks.md:164), and project hooks live in .grok/hooks/*.json, not in the project TOML (10-hooks.md:69, 197-203).
  • Release: all 9 hosts emit real config + @metaharness/harness (metaharness 0.1.12, hosts 0.1.1) #8 (open) is the metaharness 0.1.12 release note. It reports, and does not establish, that ADR-046 treats real-install execution as the top tier of host verification (docs/adrs/ADR-046-real-install-host-verification.md:91-94), implemented as an opt-in scripts/verify-all-hosts.mjs --real pass in which uninstalled runtimes report skip so CI stays green and only --real-strict fails the gate (ADR-046 :33-36; scripts/verify-all-hosts.mjs:167-176). 7 of 9 hosts were verified live; rvm and copilot are listed under "documented, not faked" (ADR-046 :55-60). Section 5 is that tier for Grok, in the same shape.

2. Current behavior at 42f568b

--host grok fails validation with Unknown host: grok. Choose from: ... and exit 2 (from source, packages/create-agent-harness/src/index.ts:1069-1071; not executed, the clone has no dist/ or node_modules/). HOSTS is the ten-host catalog at :70. Grok and xAI are absent from the clone: a case-insensitive grep for grok finds one benchmark citation of "Grok 4" in docs/research/rli-remote-labor-index.md:37 plus four files that only contain "ngrok"; xai has zero hits. packages/router and packages/bench are outside my sparse clone and were not checked.

What a claude-code scaffold does under Grok today (live evidence in section 3): CLAUDE.md, .claude/rules/*.md, .claude/skills/*/SKILL.md and .claude/commands/*.md load (VERIFIED); .claude/agents/*.md is discovered by grok inspect but was not loaded into the live session (VERIFIED, discovery only); .claude/settings.json hooks are trust-gated (VERIFIED) and its permissions are trust-gated (DOC-ONLY); the mcpServers block the templates put in .claude/settings.json (packages/create-agent-harness/templates/minimal/.claude/settings.json.tmpl:12-17, same key in every templates/vertical_*/.claude/settings.json.tmpl) has no documented Grok reader, because Grok's Claude MCP import source is ~/.claude.json only (07-mcp-servers.md:229; 05-configuration.md:397) and .claude/settings.json is documented only as a source of hooks (05-configuration.md:398; 10-hooks.md:70) and permissions (22-permissions-and-safety.md:277-306) (DOC-ONLY, not probed, see section 3); and install-mcp.sh runs claude mcp add (packages/host-claude-code/src/index.ts:126-136). Net effect: the harness MCP server never reaches Grok. That is the gap a native adapter closes.

One hazard for a partial PR: hostConfigFiles returns [] for unknown ids (packages/create-agent-harness/src/host-config.ts:263-264), so adding 'grok' to HOSTS without the case silently scaffolds a claude-shaped tree that depends on an unpublished @metaharness/host-grok (templates/minimal/package.json.tmpl:18).

Every place hosts are enumerated and grok is absent, with the change each needs:

Surface File:line Change
CLI catalog, provenance comment, usage string packages/create-agent-harness/src/index.ts:70, :66-69, :1040 add 'grok'; extend the comment; the usage string still lists four hosts (stale for existing hosts too)
CLI native-config switch packages/create-agent-harness/src/host-config.ts:109-266 is hostConfigFiles(); switch (host) spans :111-265 (case arms :112-261, default at :263-264); dependency-free by design per the header comment at :8-11 (the module has zero imports) case 'grok' after the prime-agent arm (:245-261)
Web UI type, catalog, scaffold switch, verify artefacts apps/web-ui/src/generator/types.ts:10, catalog.ts:28, scaffold.ts:408-428 (parity comment :410-411), verify.ts:87-92 union member; catalog row; byte-identical case 'grok'; .grok/config.toml in hostArtifacts; permissionsPresent (verify.ts:51-61) is JSON-only today and must learn to read the TOML
Schema and real-install gate scripts/verify-all-hosts.mjs:10, :90-108, :183-258 HOSTS entry; a checks entry for .grok/config.toml with the codex predicate (:92); a realChecks entry (section 5)
Live-LLM gate, published smoke loop scripts/verify-harness-live.mjs:33, .github/workflows/published-smoke.yml:268 add grok
Build order, version-drift exemption, tag-publish set scripts/build-ordered.mjs:47, scripts/healthcheck.mjs:130, scripts/publish-workspace.mjs:32-47 add host-grok; RELEASE_ORDER membership is your call (CODEOWNERS path, .github/CODEOWNERS:6; copilot, opencode and github-actions are absent from it today)
Bench baseline (required by CI) packages/bench/host-baseline.json via .github/workflows/ci.yml:288-294 (not in my clone) a measured row
Package keywords packages/create-agent-harness/package.json:70-76 (prime-agent at :76) add grok
Release doc docs/RELEASE.md:9-20 package table, :86-87 ("7 host adapters"), :102-111 ("10 packages") add rows; stale for existing hosts too (copilot, opencode, github-actions are missing)
Template manifests and their generator packages/create-agent-harness/templates/minimal/manifest.json:18; every templates/vertical_*/manifest.json (for example vertical_coding/manifest.json:103-108); packages/create-agent-harness/scripts/gen-templates.mjs:56 choices lists four hosts; stale for six existing hosts too
Real-runtime table docs/adrs/REVIEW-PROGRESS.md:25-35 a grok row
Dev toolkit strings scripts/dev-toolkit.mjs:124, :126 ("6 hosts") stale for existing hosts too
Example packages (optional) examples-packages/<host>/; scripts/write-example-packages.mjs:25-35 nine host directories exist, none for prime-agent; add grok only if you want the pattern kept
Docs and catalogs README.md:199 ("ten"), :201-212, :332, a feature bullet in the :72-83 shape; docs/USAGE.md:71-86; docs/USERGUIDE.md:104-112; docs/ARCHITECTURE.md:20-24; packages/create-agent-harness/README.md:92-96; .codex/skills/create-harness/skill.toml:43; docs/adrs/INDEX.md:181 shape; CHANGELOG.md:55-67 shape rows and counts

Plus a regenerated package-lock.json.

3. Grok Build CLI contract (verified)

Live method: a throwaway git repo containing every artefact below; grok inspect --json in four environments (default, all five GROK_CLAUDE_*_ENABLED=false, GROK_FOLDER_TRUST=0, from a subdirectory), the first two taken again after adding .mcp.json, .claude/agents and .claude/commands; then two headless calls with the tool lockdown shown in section 5. The first call failed after 354 s with HTTP 500 "The model is currently at capacity"; the retry completed in 164 s with stopReason: end_turn and 31,639 total tokens (session 01a067e2-5911-7e51-b535-a5eb7242a9db). VERIFIED: Grok writes the assembled model context (chat_history.jsonl), the resolved MCP set (events.jsonl) and the rendered system prompt into the session directory before the model is contacted, so "what loaded" is provable independently of the model's answer. The folder was never trusted: ~/.grok/trusted_folders.toml does not mention it, both default inspect dumps report projectTrusted: false, and the headless calls printed no trust prompt (empty stderr). The model's structured answer matched that ground truth 7 markers for 7.

Scaffold artefact Grok behaviour Guide Live result
AGENTS.md native project instructions 12-project-rules.md:17-26 VERIFIED: listed by inspect (fileType: agents_md, scope: project); marker present in the first user message inside an always_applied_workspace_rule block. inspect reports the name as Agents.md on this case-insensitive APFS volume
CLAUDE.md native, not compat-gated 12-project-rules.md:26; 05-configuration.md:407 VERIFIED: injected into the model context; still listed with every GROK_CLAUDE_*_ENABLED=false
.claude/rules/*.md compat 05-configuration.md:395 VERIFIED: injected (vendor: claude, compatibilityStatus: enabled); disabled under GROK_CLAUDE_RULES_ENABLED=false
.grok/config.toml [mcp_servers.<name>] project scope; trust-gated at connect time 07-mcp-servers.md:27-36, 69-72, 170-180; 10-hooks.md:79-81 VERIFIED discovered by inspect. VERIFIED NEGATIVE: not started in two untrusted sessions (events.jsonl mcp_config_resolved listed only the six user-level servers; the model's MCP reminder likewise). inspect attributes its source path to ~/.grok/config.toml, which is wrong (minor attribution bug)
.mcp.json loaded unless the Claude import marker is set 07-mcp-servers.md:231 VERIFIED discovered (source.type: mcpJson); VERIFIED NEGATIVE: not started untrusted
.claude/settings.json mcpServers not a documented source; the Claude MCP import reads ~/.claude.json only 07-mcp-servers.md:229; 05-configuration.md:397 DOC-ONLY. Not probed: the probe project's .claude/settings.json carried only a hooks block, so its absence from inspect mcpServers is not evidence
.grok/config.toml [permission] merged, deny > ask > allow; project rules trust-gated 22-permissions-and-safety.md:260-275, 558; 26-config-reference.md:433-436 DOC-ONLY for project rules; inspect permissions.sources listed only ~/.claude/settings.local.json (52 rules loaded)
.grok/hooks/*.json Claude three-level JSON; trust-gated 10-hooks.md:69, 79-81, 138-166 VERIFIED: absent from inspect untrusted, listed as source.type: project with GROK_FOLDER_TRUST=0. Execution: not attempted: both headless calls ran untrusted (the side-effect log was never written), and events.jsonl has no hook event type, so execution can only be proven by side effect
.claude/settings.json hooks compat; trust-gated 10-hooks.md:70 VERIFIED: same as the row above (vendor: claude)
.grok/agents/<name>.md project agent definitions 16-subagents.md:17 VERIFIED discovered as source.type: project. Spawn: not attempted: needs a second model call
.claude/agents/<name>.md undocumented none (grep -rn '\.claude/agents' over the guide is empty) VERIFIED discovered by inspect (agents[], no vendor or compatibilityStatus field, survives compat-off). NOT loaded into the live session: its marker is absent from the transcript and from the answer (agent definitions are spawn targets, not context)
.grok/skills/<name>/SKILL.md native, highest precedence 08-skills.md:21-22 VERIFIED discovered and advertised to the model with name, description and absolute path
.claude/skills/*/SKILL.md, .claude/commands/*.md compat; .gitignore ignored 08-skills.md:25, 31, 33 VERIFIED advertised to the model (both markers in the transcript and in the answer); disabled under GROK_CLAUDE_SKILLS_ENABLED=false
.codex/config.toml Codex cells "reserved and currently inert" 05-configuration.md:405 VERIFIED NEGATIVE
statusLine [ui.status_line] is user-config only 25-status-line.md:31-33; 05-configuration.md:846 DOC-ONLY, no project surface
--trust grants folder trust 10-hooks.md:79 VERIFIED absent from grok --help (no trust string in its 173 lines); grok --trust --help is accepted (exit 0, prints help). Effect: not attempted: it would write to ~/.grok/trusted_folders.toml. GROK_FOLDER_TRUST=0 (10-hooks.md:81) VERIFIED to flip projectTrusted to true in inspect
--output-format json with --json-schema structured value 14-headless-mode.md:289 says structured_output VERIFIED: the key in the json object is structuredOutput (camelCase)
Bash(git commit:*) colon form; Read(./.env) relative path suffix stripped to prefix; ./ collapsed and joined with the session cwd 22-permissions-and-safety.md:228, 325; 351-356 DOC-ONLY (both forms appear in policyLists, host-config.ts:64, and in the templates)

4. Proposed implementation

Statements in this section are DOC-ONLY unless tagged VERIFIED.

4.1 Package and emission map

Package packages/host-grok/, mirroring the newest adapter, host-prime-agent (ADR-247): package.json shape from packages/host-prime-agent/package.json:2-54; type-only import from @metaharness/kernel; HOST_NAME = 'grok' as const; pure byte-deterministic renderers; adapter and default export (packages/host-prime-agent/src/index.ts:43-45, :465-513); __tests__/fixtures.ts plus golden/default-spec.json with an LF pin in the repo-root .gitattributes (:5-6). The TOML renderer is copied from host-codex (packages/host-codex/src/index.ts:46-83), not imported (ADR-045 :66-72), and hardened per the tests below.

HarnessSpec input Emitted Notes
systemPrompt, description, agents roster AGENTS.md host-codex pattern (packages/host-codex/src/index.ts:31-40). CLAUDE.md from the template also loads (VERIFIED), so a --host grok tree carries the same text twice; the runbook says so
mcpServers[] .grok/config.toml [mcp_servers.<name>]: command = argv[0], args = rest, env = { K = "V" }, enabled = true; remote servers emit url = "..." (plus headers when present) and no type key Grok documents the HTTP form as url plus headers (07-mcp-servers.md:69-72); mcp_servers.<name>.type exists in the reference with undocumented values (26-config-reference.md:339), so the codex arm's type = "http" (host-config.ts:119, from mcpServerEntry :53-55) must not be copied (open question 1). Names normalised to [A-Za-z0-9_-] (07-mcp-servers.md:145) with a deterministic -2 suffix on collision; env keys must be TOML bare-key-safe or quoted (the codex renderer writes them unquoted, packages/host-codex/src/index.ts:71-73); committed configs should reference ${VAR} (07-mcp-servers.md:135)
permissions.allow/deny [permission] allow = [...], deny = [...] in the same file; table name singular strings pass through verbatim: the guide says the native table takes the same rule strings as .claude/settings.json, including the colon form and the mcp__server__tool spelling (22-permissions-and-safety.md:260, 225-232, 365-367) (DOC-ONLY; only user-level rules were observed loading). Absent permissions produce no table; a non-empty deny without a [permission] table is a test failure. Runtime inertness on an untrusted checkout is handled by 4.2
hooks[] .grok/hooks/<harness-name>.json rules in 4.1.1
tools[] .grok/skills/<name>/SKILL.md, instruction-only prime-agent doctrine (packages/host-prime-agent/src/index.ts:27-31); frontmatter name, description (08-skills.md:92-95). The template's .claude/skills/evolve is left alone; it loads via compat (VERIFIED for .claude/skills)
agents[] .grok/agents/<name>.md, frontmatter name, description, body = systemPrompt the native directory, because .claude/agents discovery is undocumented and could vanish
statusLine named under "Unsupported on this host" with the [ui.status_line] snippet user-config only
autonomous.maxTurns grok -p ... --max-turns N line in the runbook (14-headless-mode.md:36) goal, heartbeat, gateCommand: explicit documented no-op per packages/kernel-js/src/types.ts:41-44
always install-grok.md (host-qualified, packages/host-prime-agent/src/index.ts:47-49) opens with the trust banner when 4.2 requires it; verification lines grok inspect --json and grok mcp doctor --json (07-mcp-servers.md:128-130); warns that grok mcp list --json prints env values in cleartext (VERIFIED)

4.1.1 Hook rendering rules

  • Shape: the Claude three-level JSON (10-hooks.md:138-158). Event allowlist is the 15 documented events (10-hooks.md:89-105); Setup and FileChanged from the SDK union (packages/sdk/src/index.ts:37) and anything else are not emitted and are named in the runbook.
  • Matcher: Grok compiles it as a regular expression over the tool name (10-hooks.md:163) and aliases Claude tool names inside it, so Bash also matches run_terminal_command (10-hooks.md:168-180). metaharness's documented matcher dialect is the pseudo-DSL Bash(rm *) (docs/adrs/ADR-004-host-integration-model.md:107; packages/host-claude-code/src/index.ts:18), which as a regex never matches a tool name. Rule: pass a matcher through only when it is a bare tool name or a regex without a (...) argument list; for a Tool(args) matcher emit the tool name and list the dropped argument predicate under "Unsupported on this host" (command filtering belongs in the handler, which is what the guide's own example does at 10-hooks.md:618). Omit the key when absent; never emit * (host-claude-code emits it at :80; Grok defines only empty or omitted, 10-hooks.md:163).
  • Handlers: http(s):// becomes {type:"http",url}. Any other handler name becomes {type:"command",command:"node bin/<name>.cjs"}, a path relative to the JSON file (10-hooks.md:165; the guide's best practice is scripts in bin/ next to the JSON, 10-hooks.md:644), with the helper user-supplied on both hosts, which is what host-claude-code already does (:69 points at node .claude/helpers/<name>.cjs and never generates it). Whether an inline command resolves against the JSON file's directory or the workspace root is unverified (open question 2). mcp:, prompt:, agent: prefixes (packages/host-claude-code/src/index.ts:52-70) have no Grok handler type (10-hooks.md:164) and are named as unsupported. Every handler gets an explicit timeout (default 5 s, 10-hooks.md:166).
  • Runbook note for helpers shared with a Claude Code scaffold: Grok's stdin envelope is camelCase (toolInput, stopHookActive), with hook_event_name and tool_response snake-case aliases (10-hooks.md:256, 446-447).

4.2 Trust posture (the fail-closed rule)

Project [permission] rules in .grok/config.toml and .claude/settings.json are skipped on an untrusted checkout (22-permissions-and-safety.md:558), project hooks are silently skipped (10-hooks.md:79), project MCP servers are not started (VERIFIED NEGATIVE), and a fresh scaffold is untrusted (VERIFIED: projectTrusted: false in every default inspect). So an emitted deny-list is inert at runtime until someone trusts the folder, which is the ADR-046 bug class that ADR-247 :48-54 makes the load-bearing rule of a host ADR. Grok's version of that rule:

  1. When permissions.deny, hooks or mcpServers is non-empty, install-grok.md opens with an ACTION REQUIRED banner stating that project [permission] rules, .grok/hooks/*.json and project [mcp_servers] are skipped until the folder is trusted, and that trust is granted by /hooks-trust in the TUI or lifted with GROK_FOLDER_TRUST=0 for CI (--trust is documented, hidden from --help, and its effect is unverified).
  2. The runbook's headless invocation repeats every deny entry as a --deny <RULE> flag: CLI rules "are always enforced" (22-permissions-and-safety.md:223), take the same rule strings (:260), and exist on 1.0.17 (grok --help lines 34-35, VERIFIED).
  3. Test: banner present iff hooks.length || mcpServers.length || deny.length, absent otherwise (mirror packages/host-prime-agent/__tests__/index.test.ts:106-135).

4.3 Three codegen paths

Besides the adapter, add case 'grok' to packages/create-agent-harness/src/host-config.ts after the prime-agent arm (:245-261), emitting .grok/config.toml built from mcpServerEntry/policyLists (:51-67) plus AGENTS.md and install-grok.md, and the byte-identical case in apps/web-ui/src/generator/scaffold.ts after :408-428 (parity comment :410-411). In all three paths the remote form emits url only, dropping the type: 'http' that mcpServerEntry returns at :53-55. Both land in the same PR as the HOSTS edit.

4.4 Docs and ADR

New docs/adrs/ADR-278-host-grok.md (278 is the next free number at 42f568b, highest file ADR-277-autogenous-metaharness-adapter.md; re-check before the PR since the series is shared with meta-llm), header and sections per ADR-247 (:1-13, Emission map :35-46, fail-closed rule :48-54, Alternatives Considered :67-73, numbered Test Contract :75-85), stating "11th implemented adapter" the way ADR-247 :91 counts. INDEX.md row in the :181 shape; README.md:199 ("ten"), :201-212, :332, plus a feature bullet in the :72-83 shape; docs/USAGE.md, docs/USERGUIDE.md, docs/ARCHITECTURE.md, packages/create-agent-harness/README.md; CHANGELOG.md bullet in the :55-67 form; docs/adrs/REVIEW-PROGRESS.md:25-35 row.

5. Real-install check (ADR-046 tier)

Two stages for verify-all-hosts.mjs --real. Skip when grok is not on PATH, and skip Stage B when not logged in; every skip carries a proof string, the "documented, not faked" shape (ADR-046 :55-60).

Stage A, no model call, under a throwaway GROK_HOME (05-configuration.md:801), the pattern the codex and hermes arms use with CODEX_HOME and HERMES_HOME (scripts/verify-all-hosts.mjs:199-211, :235-245), which also keeps the user's compat MCP servers and config warnings out of the picture:

cd <scaffold> && GROK_HOME=<tmp> GROK_FOLDER_TRUST=0 grok inspect --json

Assert: projectInstructions[] has an entry whose path ends with agents.md compared case-insensitively (my inspect reports Agents.md on APFS) with scope: project; mcpServers[] contains the harness server; hooks[] has a source.type: project entry when hooks were emitted; agents[] contains each emitted agent; no configWarnings entry whose path points inside the scaffold (do not assert the array is empty: on my machine a stale user-config key yields one unknown-field warning); record grokVersion. Without GROK_FOLDER_TRUST=0 the hooks assertion fails by design. Not yet checked: whether grok inspect runs cleanly with an empty GROK_HOME (open question 3).

Stage B, one model call against the real home (auth.json lives in GROK_HOME, 05-configuration.md:823), with the tool lockdown that completed in my probe (only the prompt, --max-turns, --effort and the two environment variables differ):

GROK_FOLDER_TRUST=0 GROK_CLAUDE_MCPS_ENABLED=false \
grok -p "Reply with exactly: REAL_OK" --cwd <scaffold> --output-format json --max-turns 1 --effort low \
  --permission-mode dontAsk --tools read_file \
  --disallowed-tools "run_terminal_cmd,run_terminal_command,monitor,workflow,search_replace,write_file,todo_write,spawn_subagent,memory_search,search_tool,use_tool,web_search,web_fetch,Agent,Agent(explore),Agent(plan)" \
  --deny Bash --deny Edit --deny Write --deny 'MCPTool(*)'
  • GROK_FOLDER_TRUST=0 makes the scaffold's project hooks and [permission] rules live. Before the call, either flip the harness server's enabled to false in the scaffold's .grok/config.toml so the unpublished npx <name>@latest mcp start does not spawn (the opencode arm does the same at :220-225), or accept the 30 s startup timeout (07-mcp-servers.md:33; a failed MCP handshake is recorded, not fatal, VERIFIED).
  • GROK_CLAUDE_MCPS_ENABLED=false (26-config-reference.md:119): 23,871 of my 31,639 tokens were input, dominated by 502 tool schemas from the user's compat MCP servers; --deny 'MCPTool(*)' gates calls but does not stop schema loading.
  • --effort low: my probe inherited the config default xhigh and waited 91 s for the first token. Whether low is on grok-4.6's menu is DOC-ONLY (14-headless-mode.md:37).
  • Assert JSON.parse(stdout).text.includes('REAL_OK'), not equality: the user's global instruction files are in the model context (VERIFIED).
  • Hook execution: put a SessionStart hook in the scaffold that appends to a temp file and assert the file exists after the call. Execution can only be proven by side effect (no hook event type in events.jsonl, VERIFIED), and it is the one thing my probes never exercised.
  • Failure handling: retry once, then record skip with the HTTP status as proof. The claude-code arm retries once for a different reason, nested-CLI contention (scripts/verify-all-hosts.mjs:186-190); the Grok cause observed on 2026-09-03 was server capacity (354 s to HTTP 500).
  • The --disallowed-tools list names both run_terminal_cmd and run_terminal_command because the guide uses the first for --tools (14-headless-mode.md:53) and the second in hooks (10-hooks.md:172); strings on the 1.0.17 binary finds both identifiers (VERIFIED), so both are passed until upstream reconciles the id (open question 6).

6. Alternatives considered

  1. Compat-only reuse of the claude-code tree. Rejected: the mcpServers block in .claude/settings.json has no documented Grok reader and install-mcp.sh calls claude mcp add, so the harness server never arrives; hooks and permissions are trust-gated either way; and every compat cell can be switched off by user config or environment (26-config-reference.md:117-121), so a scaffold that depends on compat can be silently disabled.
  2. Emit .mcp.json. Rejected: loading depends on the per-user Claude import marker (07-mcp-servers.md:231), and it is trust-gated anyway (VERIFIED NEGATIVE).
  3. An install-mcp.sh of grok mcp add --scope project. Rejected as redundant: it writes the same .grok/config.toml (07-mcp-servers.md:135) and adds a manual step.
  4. Package the harness as a Grok plugin for --plugin-dir. Kept as open question 4: the flag is auto-trusted ("hooks and MCP servers activate without a prompt", grok agent --help lines 23-24) but exists only on grok agent; the root command rejects it (error: unexpected argument '--plugin-dir' found, VERIFIED), so it does not help a -p runbook or Stage B as written.

7. Open questions

  1. Does Grok warn, ignore or reject type = "http" inside [mcp_servers.*]?
  2. Is a relative hook command resolved against the JSON file's directory (10-hooks.md:165) or the workspace root?
  3. Does grok inspect --json run cleanly with an empty throwaway GROK_HOME?
  4. Is grok agent --plugin-dir a viable CI path for a scaffold's hooks and MCP server?
  5. Does the hidden --trust flag persist trust in headless -p mode?
  6. Which shell tool id does a hook see on stdin, run_terminal_cmd or run_terminal_command?
  7. What does a hook with a * matcher do (host-claude-code emits it at :80)?

Tests

  • Identity: HOST_NAME === 'grok', adapter.name === 'grok'.
  • serverToToml: stdio and url forms; env inline table; enabled = true; a server with neither command nor url emits nothing; no type = line anywhere; name normalisation and -2 suffixing are deterministic; an adversarial name containing ], a newline and a space normalises without producing a second table header; env keys that are not bare-key-safe are quoted (ADR-046 :142-147 adversarial-input requirement).
  • [permission]: allow/deny round-trip verbatim; absent permissions produce no table; non-empty deny with empty allow is still emitted.
  • Hooks JSON parses; the 15 documented events pass through; Setup, FileChanged, PermissionRequest are absent from the JSON and present in the runbook; http(s):// maps to http; mcp:/prompt:/agent: are absent from the JSON and present in the runbook; Bash(rm *) in gives Bash out and the runbook names the dropped predicate; no "matcher": "*" anywhere; every handler has a numeric timeout.
  • AGENTS.md emitted iff systemPrompt || description; agents roster present when agents is non-empty.
  • One instruction-only SKILL.md per tool; name matches ^[a-z0-9-]{1,64}$; description truncation is surrogate-safe (prime-agent tests :317-325).
  • .grok/agents/<name>.md per agent with sanitised frontmatter.
  • statusLine present produces the unsupported section; none of the unsupported inputs produces no section.
  • Golden file byte-equality via stableStringify, and two-call determinism (prime-agent tests :98-103, :188-195).
  • Trust banner present iff hooks.length || mcpServers.length || deny.length; the runbook's headless line carries one --deny per deny entry.
  • it.skipIf(!hasGrok): write the emitted files to a tmp git repo, run GROK_HOME=<tmp> GROK_FOLDER_TRUST=0 grok inspect --json, assert the server, agent and hook entries appear (zero model cost; hook execution stays in Stage B).
  • CLI: packages/create-agent-harness/__tests__/host-config.test.ts gains a grok signature case and an adversarial-name case; apps/web-ui/src/generator/__tests__/scaffold.test.ts:52-57 gains its assertion; packages/create-agent-harness/__tests__/integration/multi-host.test.ts:16 picks up grok automatically.

Acceptance criteria

  • npx metaharness my-bot --template minimal --host grok succeeds and writes .grok/config.toml, AGENTS.md, install-grok.md.
  • Adapter, host-config.ts and scaffold.ts produce byte-identical .grok/config.toml and AGENTS.md for the same input, with url only for remote servers.
  • node scripts/verify-all-hosts.mjs PASSES for grok; --real PASSES Stage A on a machine with grok 1.0.17 and Stage B when logged in, and otherwise SKIPS with a proof string (documented, not faked).
  • scripts/healthcheck.mjs HEALTHY; published-smoke.yml loop scaffolds bot-grok.
  • No HarnessSpec field is silently dropped, at emission time or at runtime: each unsupported input is named in install-grok.md, the trust banner and the --deny flags cover a deny-list that the untrusted checkout would otherwise ignore, and each case is covered by a test.
  • install-grok.md states the trust gate and the --trust/GROK_FOLDER_TRUST=0 situation exactly as verified above.
  • ADR-278 (or the next free number) added with the pinned version string, the VERIFIED/DOC-ONLY table and an Alternatives Considered section; INDEX, README, USAGE, USERGUIDE, ARCHITECTURE, CHANGELOG, REVIEW-PROGRESS rows updated; README count says eleven.
  • Decision recorded on RELEASE_ORDER membership.

Evidence bundle

Gist: https://gist.github.com/HF-teamdev/5d9d997daf5438788e858f6f8f55d098

  • 01-grok-help.txt: grok --help on 1.0.17 (173 lines).
  • 02-probe-project-tree.md: the probe project's file tree with the marker each file carried, and the run order.
  • 03-headless-call-2-result.json: sessionId, stopReason, usage and the structuredOutput of the successful call (7 markers echoed); 03a is the HTTP 500 body from the first attempt.
  • 04-events-mcp_config_resolved.json: the mcp_config_resolved event from both sessions' events.jsonl, reduced to server names and transports: six user-level servers, no mh-probe.
  • 05-inspect-excerpts.json: grok inspect --json for the default, compat-off, GROK_FOLDER_TRUST=0 and subdirectory runs, limited to the probe project's entries in projectInstructions, skills, agents, hooks and mcpServers, with a count of omitted user-level entries per array.

Redaction: env and headers values are stripped and local paths are neutralised, since inspect and grok mcp list --json can carry them.

Environment

  • grok 1.0.17 (a549186d9d39) [alpha], ~/.grok/bin/grok, macOS (Darwin 25.5.0, arm64, case-insensitive APFS), grok.com session login, no XAI_API_KEY
  • ruvnet/metaharness at 42f568b7297c59065ea562247937bb25cd353a6a (committed 2026-09-02 02:31 -0400; shallow sparse-checkout --filter=blob:none clone made 2026-09-03)
  • Date of probes: 2026-09-03

Offer

I can re-run any of the probes above, run the Stage B hook-execution probe with GROK_FOLDER_TRUST=0, or test a PR branch, on macOS with this Grok build.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions