Prevent Metal watchdog failures during long-context prefill

[m21ecom]

Summary

Prevent long-context prompt prefill from submitting Metal attention work large enough to trigger MTLCommandBufferErrorDomain Code=1 / Impacting Interactivity.

Changes

• Split long full-attention work into bounded query spans.
• Adapt the query batch size as KV length grows.
• Add --prefill-watchdog-protection on|off, enabled by default.
• Preserve the original unbounded path when protection is disabled.
• Add labeled Metal command-buffer errors with layer, query range, and KV context.
• Log periodic prefill and decode progress for long-running server requests.
• Document the new server and CLI control.

Motivation

Long prompts were failing for me during prefill with errors such as:

Metal command buffer failed: label=prefill ...
MTLCommandBufferErrorDomain Code=1
Impacting Interactivity

A fixed eight-query batch was still too large around KV length 6,784. The protected path now reduces the batch size with KV length—for example, approximately three rows around 6.8K KV, two around 10K, and one near 32K.
Validation
Environment:

• Apple M2 Pro, 16 GB
• macOS 26.6.2
• Swift 6.3.3
  Results:
• Release server build passed.
• All 698 package tests passed.
• A 10,229-token prompt completed successfully with --max-context 32768.
• Two fresh-server 5,000-token runs per mode showed no measurable performance penalty:
  • Protection off mean: 129.017 seconds
  • Protection on mean: 128.214 seconds
  • Difference: −0.62%, within normal run variance
    The 32K validation means the server was configured for a 32K context window; the tested prompt itself contained 10,229 tokens.