[FEATURE] MCP client for Foreman agents (docs + search grounding)

[Defilan]

Feature Description

Give Foreman's coder and reviewer agents a built-in MCP (Model Context
Protocol) client so they can look things up at run time (documentation via
context7, web search via Brave) instead of relying solely on the model's
training memory. This is an epic tracking the capability across phases.

Problem Statement

Foreman agents hallucinate facts that live outside the model: exact metric
names, current API groups, library function signatures, config syntax. Recent
runs show the pattern concretely:

• #409 — coder invented a vLLM metric/label (vllm:request_failure_total)
  that does not exist.
• #850 — coder wrote the wrong API group (inference.defilante.tech instead
  of inference.llmkube.dev).
• #809 — coder fabricated behavior it could have grounded against real docs.

As a Foreman operator, I want my agents to consult authoritative sources
(docs, search) during a run, so that they ground claims in real information
and stop confidently emitting wrong names/syntax.

The same instinct works well interactively: an agent with a docs/search MCP
server makes up far less because it can go read the real thing.

Proposed Solution

An in-agent MCP client, wired into the existing tool loop so MCP tools appear
as ordinary tools named mcp/<server>/<tool>, subject to the same
Agent.spec.tools whitelist, loop, and transcript as native tools. Opt-in per
agent; opt-out per Workload (for clean benchmark control runs).

apiVersion: foreman.llmkube.dev/v1alpha1
kind: Agent
spec:
  role: coder
  tools: [read_file, write_file, str_replace, grep, bash, fetch_issue, submit_result]
  mcp:
    enabled: true                 # per-agent gate; default off (opt-in)
    callTimeout: 30s
    maxResultBytes: 32768
    servers:
      - name: context7
        transport: http
        url: https://mcp.context7.com/mcp
        allowedTools: ["*"]

Phases

• Phase 1 — HTTP client, context7, coder+reviewer (opt-in). MCP client
  over the official Go SDK (HTTP/streamable transport); tool adapter; Connect
  entry point; CRD fields (Agent.spec.mcp, Workload.spec.mcpEnabled,
  AgenticTask.spec.mcpEnabled); wiring in the tool-registry factory;
  secret-sourced auth headers; best-effort (a server/secret failure is logged
  and skipped, never fails a run); UTF-8-safe result cap; per-call timeout.
  Audit is the existing run transcript. ← this PR.
• Phase 2 — Brave web search via an adopted, well-supported OSS
  stdio→HTTP bridge (e.g. mcp-proxy / supergateway), so search-capable stdio
  servers are reachable over the same HTTP path.
• Phase 3 — native stdio transport in the client (drop the bridge for
  local stdio servers), plus streaming/large-result handling as needed.

Validation

A/B the MCP-enabled coder against the un-MCP baseline on #409 and #850:
success = the MCP-on run cites the real vLLM metric and the correct
inference.llmkube.dev API group where the MCP-off run hallucinated.

Alternatives Considered

• Prompt-only grounding (paste docs into the system prompt): stale, bloats
  context, doesn't scale across topics.
• A bespoke docs/search tool instead of MCP: reinvents an ecosystem that
  already exists; MCP lets us adopt maintained servers (context7, Brave) rather
  than build and maintain fetchers ourselves.
• RAG index of docs: heavier infra; MCP servers already do retrieval and
  stay current without us maintaining an index.

Additional Context

• Related issues: feat(metrics): expose TTFT and per-request error rate; enable vLLM /metrics #409, [BUG] hostpath PVC provisioning blocked on tainted GPU nodes (breaks AMD/Vulkan model staging) #850, [FEATURE] Reviewer issueAsk verification: semantic coverage, not verbatim quote #809 (hallucinations this grounds against).
• Design + implementation plan authored before build (kept in a private working
  repo); this epic captures the public-facing summary.
• Security posture: read-only tools only in Phase 1; headers only from Secrets;
  MCP results are treated as untrusted model-facing data; results size-capped.

Priority

• High - Would significantly improve my workflow

Willingness to Contribute

• Yes, I can submit a PR