Track intermittent MCP bridge-ready CI timeouts

[ejgallego]

Summary

We have seen intermittent broker-fast / MCP stdio timeout failures on unrelated PRs. The failure appears scheduler-sensitive, with macOS / low-core CI workers suspected as a useful reproduction hint.

Use this issue as the shared incident log so unrelated PRs can link their occurrences instead of diagnosing the race independently.

What to comment when this happens

When an unrelated PR hits this race, comment with:

• PR URL and branch
• failing GitHub Actions run URL and failing job URL
• job name, runner OS/arch, run attempt number, and commit SHA
• failing test or scenario, for example test-beam-fast.sh, test-mcp-stdio.py, progress-smoke, or save_olean sentinel
• timeout headline from the CI log
• MCP stdio diagnostic excerpt when present: pending requests, completed requests, server requests, recent notifications, event timeline, stderr tail, server trace or diagnostics-barrier watchdog lines, and process snapshot
• save-replay diagnostic excerpt when present: sentinel state, daemon registry, daemon log tail, save stdout/stderr, diagnostics-barrier watchdog lines, and process snapshot
• rerun URL and whether the rerun passed or reproduced

Comment template

### Occurrence

- PR: <url>
- Branch: <branch>
- Commit: <sha>
- Run: <url>
- Job: <url>
- Job name / OS / arch / attempt: <...>
- Failing test or scenario: <...>
- Rerun: <url or none yet>, <passed/reproduced/pending>

### Timeout headline

```text
<paste headline>
```

### Diagnostic excerpt

```text
<paste the relevant MCP stdio or save sentinel sections>
```

Current clues

• test: improve MCP stdio timeout diagnostics #95 added richer timeout diagnostics for MCP stdio and save-replay sentinel failures.
• The likely failure family involves broker/MCP progress under scheduler pressure, not the unrelated PR changes that happen to expose it.
• Low-core/macOS CI workers may be the strongest reproduction hint.

Non-goals

Do not use this issue for ordinary Lean build failures or sync/readiness semantic bugs. Those should stay on their own issues, such as #99 for saveReady correctness.

[ejgallego]

Observed on PR #105 (codex/claude-config-path), run attempt 2. This PR is installer/config ergonomics; the installer jobs passed on Ubuntu and macOS, and the reproduced failures are both macOS bridge-ready timeouts.

Run URL: https://github.com/ejgallego/lean-beam/actions/runs/28115853356

Jobs:

• mcp-conformance (macos-latest), attempt 2, job 83265844976: https://github.com/ejgallego/lean-beam/actions/runs/28115853356/job/83265844976
• beam-fast (macos-latest), attempt 2, job 83265845380: https://github.com/ejgallego/lean-beam/actions/runs/28115853356/job/83265845380

Runner details for both jobs:

• OS: macOS 26.4 (25E246)
• image: macos-26-arm64, version 20260610.0145.2
• Azure region: westus

Timeout headlines / excerpts:

mcp-conformance:

• build MCP server completed successfully: Build completed successfully (113 jobs).
• scenario: server-initialize
• timed out waiting for MCP bridge ready file
• scenario finished in 30s

beam-fast:

• command: bash tests/test-beam-fast.sh
• test-mcp-http-bridge.py: timed out waiting for bridge ready file /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/lean-beam-mcp-http-zrfjqp4l/ready.json

Rerun result: reproduced on both failed macOS jobs. All non-bridge checks, including beam-install on both Ubuntu and macOS, passed.

[ejgallego]

Occurrence

• PR: test: point CI timeout failures at tracker #111
• Branch: codex/ci-timeout-issue-hint
• Commit: e499058811f5da247ab645cc41bc21cc072fad8e
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28134957825
• Failing job: https://github.com/ejgallego/lean-beam/actions/runs/28134957825/job/83319595809
• Job name / OS / arch / attempt: mcp-conformance (macos-latest), macos-latest, arch not reported by job metadata, attempt 1
• Runner: GitHub Actions 1000018152, labels: macos-latest
• Failing test or scenario: mcp-conformance, scenario server-initialize
• Rerun: https://github.com/ejgallego/lean-beam/actions/runs/28134957825/job/83321288149, attempt 2 passed in 1m57s

Timeout headline

timed out waiting for MCP bridge ready file

Diagnostic excerpt

Build completed successfully (113 jobs).
[mcp-conformance] build MCP server finished in 54s
scenario: server-initialize
[mcp-conformance] scenario: server-initialize
timed out waiting for MCP bridge ready file
[mcp-conformance] scenario: server-initialize finished in 30s
Process completed with exit code 1.

[huangruiteng]

Hi, I operate LoopX, an agent-ops system for taking recurring CI and automation stalls and running bounded evidence loops until an owner decision exists.

The incident log here is already strong: each occurrence captures PR, run, job, runner, scenario, timeout headline, and rerun result. The next useful contribution is probably not another static analysis, but taking over a bounded public occurrence-to-decision loop.

If helpful, LoopX can own that loop here using only public issue/run links:

• track future broker-fast / MCP stdio timeout occurrences;
• maintain a matrix by runner image, scenario, and rerun outcome;
• compare the "macOS/low-core scheduler pressure" and "bridge startup/readiness race" hypotheses;
• propose the next minimal probe, watchdog, timeout-policy change, or job narrowing when evidence is enough;
• stop when the project chooses a guardrail, the issue closes, or the next step requires non-public runner data.

Would that public experiment loop be useful here?

LoopX repo: https://github.com/huangruiteng/loopx

[ejgallego]

PR/run: #113 / https://github.com/ejgallego/lean-beam/actions/runs/28137978797

Head: codex/install-test-helpers-deps-readiness @ f7b1b9d
Attempt: 1

Failing jobs:

• mcp-conformance (macos-latest), job 83328862283
  • OS/image: macOS 26.4, macos-26-arm64, image version 20260623.0192.1
  • Timeout: scenario: server-initialize timed out waiting for MCP bridge ready file after 30s.
  • No request summary / notification / trace lines were present in the job log beyond the ready-file timeout headline.
• beam-fast (macos-latest), job 83328862303
  • OS/image: macOS 26.4, macos-26-arm64, image version 20260610.0145.2
  • Timeout: test-mcp-http-bridge.py timed out waiting for bridge ready file /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/lean-beam-mcp-http-abtprx4j/ready.json.
  • No request summary / notification / trace lines were present in the job log beyond the ready-file timeout headline.

All other checks in the run passed. This matches the tracked MCP bridge ready-file timeout class.

[ejgallego]

PR #113 rerun update for https://github.com/ejgallego/lean-beam/actions/runs/28137978797:

• mcp-conformance (macos-latest) rerun job 83401466762 passed.
• beam-fast (macos-latest) rerun job 83401466748 failed again on macOS 26.4 / macos-26-arm64, image version 20260623.0192.1.
• Timeout headline: test-mcp-http-bridge.py timed out waiting for bridge ready file /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/lean-beam-mcp-http-t71rzenu/ready.json.
• No request summary / notification / trace lines were present in the rerun job log beyond the ready-file timeout headline.

[ejgallego]

Occurrence

• PR: refactor: split installer location helpers #114
• Branch: codex/install-location-refactor
• Commit: de9f927481d6523fd95315759ef4cf61feeb855c
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28168998922
• Job: https://github.com/ejgallego/lean-beam/actions/runs/28168998922/job/83428078650
• Job name / OS / arch / attempt: mcp-conformance (macos-latest), macOS 26.4, macos-26-arm64, attempt 1
• Failing test or scenario: tests/test-mcp-conformance.sh, scenario server-initialize
• Rerun: none; PR merged after this non-required macOS timeout, while the branch-relevant installer checks passed

Timeout headline

timed out waiting for MCP bridge ready file

Diagnostic excerpt

[mcp-conformance] build MCP server finished in 77s
[mcp-conformance] scenario: server-initialize
timed out waiting for MCP bridge ready file
[mcp-conformance] scenario: server-initialize finished in 30s

[ejgallego]

CI timeout occurrence on PR #112 (codex/mcp-bridge-ci-diagnostics).

PR: #112
Branch: codex/mcp-bridge-ci-diagnostics
Commit: 5f3c23bc2a7abc5ed1756bdbd78d557115d9cd72

Failing run: https://github.com/ejgallego/lean-beam/actions/runs/28168899009
Failing job: https://github.com/ejgallego/lean-beam/actions/runs/28168899009/job/83427740453
Job: beam-fast (ubuntu-latest)
Run attempt: 1
Runner: Linux X64, GitHub Actions 1000018398, ImageOS=ubuntu24
Runtime context: CPython 3.12.3, os.cpu_count=4, sched_getaffinity=4, loadavg 2.35 1.39 0.55

Timeout headline:
test-mcp-stdio.py: timed out waiting for MCP response id 5 (tools/call lean_sync (path="CommandA.lean", progressToken="sync-progress-token")) for client progress-explicit-root after 60.011s

MCP stdio state:

• Pending request: id 5, tools/call lean_sync, path="CommandA.lean", progressToken="sync-progress-token", age 60.020s.
• Recent completed requests: id 1 initialize in 0.036s; id 2 lean_sync PositionEmptyLine.lean progressToken=true in 0.000s; id 3 lean_sync PositionEmptyLine.lean progressToken=1.5 in 0.000s; id 4 lean_sync PositionEmptyLine.lean in 0.361s.
• Recent server requests received from lean-beam-mcp: none.
• Recent notifications for token sync-progress-token: progress 1 starting lean_sync, 2 preparing lean_sync, 3 running lean_sync, 4 lean_sync fileProgress line=1/1 updates=1 done=false, 5 updates=13 done=true, 6 updates=14 done=true; last notification was 59.695s before timeout.
• Broker/server trace tail shows the broker completed id 5: sync_file barrier completed, saveReadiness response matched, sync_file response ready, dispatch complete op=sync_file clientRequestId=mcp:5 ok=true, tools/call broker returned id=5 tool=lean_sync ok=true, and tools/call response ready id=5 tool=lean_sync.
• Process snapshot included lean-beam-mcp pid 6527, Lean server pid 6532, and workers for PositionEmptyLine.lean and CommandA.lean.

Rerun: https://github.com/ejgallego/lean-beam/actions/runs/28168899009/job/83431864136
Rerun result: passed on run attempt 2 (beam-fast (ubuntu-latest), 4m31s).

Local assessment: this hit matches the scheduler-sensitive stdio response timeout tracked here and is unrelated to the MCP HTTP bridge getfqdn startup fix in PR #112.

[ejgallego]

Occurrence

• PR: refactor: document save loading and reuse path helpers #116
• Branch: codex/save-docs-path-dedup
• Commit: 36346a664ef124e11ca82f82338c36f3eb5d3950
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28173781892
• Job: https://github.com/ejgallego/lean-beam/actions/runs/28173781892/job/83444544164
• Job name / OS / arch / attempt: beam-fast (macos-latest), macOS 26.4, macos-26-arm64, attempt 1
• Failing test or scenario: tests/test-beam-fast.sh / test-mcp-http-bridge.py
• Rerun: https://github.com/ejgallego/lean-beam/actions/runs/28173781892, attempt 2 pending

Timeout headline

test-mcp-http-bridge.py: timed out waiting for bridge ready file /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/lean-beam-mcp-http-okmlj0b6/ready.json

Diagnostic excerpt

Runner: macOS 26.4 (25E246), image macos-26-arm64, version 20260610.0145.2, Azure region westus.
Command: bash tests/test-beam-fast.sh
Only the bridge ready-file timeout headline was present in the job log; all other #116 checks passed on attempt 1.

[ejgallego]

Occurrence

• PR: test: improve toolchain compat diagnostics #118
• Branch: codex/install-fixture-sharedlib-dedupe
• Commit: 2b9e4e56b07acce0cbbec61e41ee6025092cd10e
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28181511135
• Job: https://github.com/ejgallego/lean-beam/actions/runs/28181511135/job/83472122515
• Job name / OS / arch / attempt: beam-fast (macos-latest), macOS 26.4, macos-26-arm64, attempt 1
• Failing test or scenario: tests/test-beam-fast.sh / test-mcp-http-bridge.py
• Rerun: none; targeted job rerun was refused while the workflow was still active, and PR test: improve toolchain compat diagnostics #118 was merged after this matched the known non-required bridge-ready timeout class.

Timeout headline

test-mcp-http-bridge.py: timed out waiting for bridge ready file /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/lean-beam-mcp-http-plv0mqu_/ready.json

Diagnostic excerpt

Runner: macOS 26.4 (25E246), image macos-26-arm64, version 20260610.0145.2, Azure region westus.
Command: bash tests/test-beam-fast.sh
Only the bridge ready-file timeout headline was present in the job log. Ubuntu lanes passed; macOS mcp-conformance and beam-slow also passed before merge.

[ejgallego]

Occurrence

• PR: refactor: share Lean module path helper #117
• Commit: 5f996b0436c3c60056142dc3e2ff8af16e3f60cc
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28182110281
• Attempt 1:
  • mcp-conformance (macos-latest) failed: https://github.com/ejgallego/lean-beam/actions/runs/28182110281/job/83474319096
  • beam-fast (macos-latest) failed in test-mcp-http-bridge.py: https://github.com/ejgallego/lean-beam/actions/runs/28182110281/job/83474319141
• Attempt 2:
  • beam-fast (macos-latest) passed: https://github.com/ejgallego/lean-beam/actions/runs/28182110281/job/83485747196
  • mcp-conformance (macos-latest) reproduced: https://github.com/ejgallego/lean-beam/actions/runs/28182110281/job/83485747217
• Runner: macOS 26.4, macos-26-arm64
• Failure: MCP bridge ready-file timeout after successful build, scenario server-initialize.

Build completed successfully (113 jobs).
[mcp-conformance] scenario: server-initialize
timed out waiting for MCP bridge ready file

[ejgallego]

Occurrence

• PR: fix: return accepted version on stale document errors #141
• Branch: codex/version-mismatch-current-version
• Commit: 8f990fbfcbc91b24a42df1e4998cc0688421aab5
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28371673497
• Job: https://github.com/ejgallego/lean-beam/actions/runs/28371673497/job/84050765540
• Job name / OS / arch / attempt: beam-fast (ubuntu-latest), labels ubuntu-latest, runner GitHub Actions 1000020668, attempt 1
• Failing test or scenario: tests/test-beam-fast.sh / test-mcp-http-bridge.py
• Rerun: https://github.com/ejgallego/lean-beam/actions/runs/28371673497/job/84060491438, attempt 2 passed in 4m40s

Timeout headline

test-mcp-http-bridge.py: timed out

Diagnostic excerpt

beam-fast (ubuntu-latest)  Beam Fast Test  2026-06-29T12:23:13Z Run bash tests/test-beam-fast.sh
beam-fast (ubuntu-latest)  Beam Fast Test  2026-06-29T12:27:04Z test-mcp-http-bridge.py: timed out
Process completed with exit code 1.

This occurrence is the HTTP-bridge smoke surface rather than direct test-mcp-stdio.py, but it is being logged here because it matches the same intermittent broker-fast / MCP bridge timeout family. The rerun passed, and all other PR checks were green.

[ejgallego]

Occurrence

• PR: doc: align README with alpha announcement #54
• Branch: codex/docs-human-review
• Commit: 4ace72ed2b3d56b200851d6921980f0f99638a17
• Run: https://github.com/ejgallego/lean-beam/actions/runs/28379662482
• Job: https://github.com/ejgallego/lean-beam/actions/runs/28379662482/job/84078532974
• Job name / OS / arch / attempt: beam-fast (ubuntu-latest), Linux X64, runner GitHub Actions 1000020892, ImageOS=ubuntu24, attempt 1
• Failing test or scenario: tests/test-beam-fast.sh / test-mcp-stdio.py, client progress-explicit-root
• Rerun: failed-only rerun was attempted during investigation, but GitHub refused it because the workflow was still active; beam-install (macos-latest) was still in progress at that point.

Timeout headline

test-mcp-stdio.py: timed out waiting for MCP response id 5 (tools/call lean_sync (path="CommandA.lean", progressToken="sync-progress-token")) for client progress-explicit-root after 60.052s

Diagnostic excerpt

Runtime context: CPython on Linux X64, os.cpu_count=4, sched_getaffinity=4, loadavg 2.25 1.34 0.53.
Pending request: id 5, tools/call lean_sync, path="CommandA.lean", progressToken="sync-progress-token", age=60.064s.
Recent completed requests: initialize in 0.032s; lean_sync PositionEmptyLine.lean with progressToken=true in 0.000s; progressToken=1.5 in 0.000s; plain lean_sync PositionEmptyLine.lean in 0.436s.
Recent server requests from lean-beam-mcp: none.
Recent notifications for sync-progress-token reached progress 6, with fileProgress rangeEndLine=1 updates=11 done=true; last notification was 59.720s before timeout.
Server trace shows the broker completed id 5: sync_file barrier completed, saveReadiness response matched, sync_file response ready, dispatch complete op=sync_file clientRequestId=mcp:5 ok=true, tools/call broker returned id=5 tool=lean_sync ok=true, and tools/call response ready id=5 tool=lean_sync.

Local assessment: this PR changes documentation and skill text only, so this occurrence appears unrelated to the branch diff and matches the tracked scheduler-sensitive MCP stdio response timeout family.