Description
The OS Matrix — compile + test workflow intermittently fails its test phase on the Linux container legs. It is flaky: a leg that fails usually passes on re-run. The signature is the TypeScript client integration suite losing its WebSocket connection to the engine's task service (ECONNREFUSED on the dynamic /task/service port), followed by a jest worker failing to exit.
The C++ engine is not crashing. A minidump recovered during a failing run (ubuntu:24.04, run 29639419996) was symbolicated and turned out to be the deliberate crash-handling test's child process (RR_CRASH_CHILD=crash, the intentional null-deref at engine-core/test/testMain.ipp:106). Crashpad catching that is the minidump-recovery test working as designed — it is a red herring, not the fault. This is also separate from the already-fixed stdout/uv-probe corruption (commit 4c7b0c76).
Steps to Reproduce
- Dispatch
OS Matrix — compile + test on feat/crashpad (Actions ▸ Run workflow).
- Let the container legs run the
Unit tests step.
- Observe one or more legs fail intermittently in the TS client integration suite; re-running the same commit usually turns the failed leg green.
Expected vs Actual Behavior
Expected: the test phase passes deterministically for a given commit.
Actual: the TS client suite (packages/client-typescript/tests/RocketRideClient.test.ts) intermittently cannot reach the engine's dynamic task-service WebSocket port — connect ECONNREFUSED — and a jest worker then fails to exit gracefully.
Severity
P2 - Medium (feature impaired, workaround exists) — flaky; a re-run usually passes.
Platform
Version / Environment
feat/crashpad (PR #1524, breakpad→crashpad migration) · OS Matrix workflow · ubuntu:24.04 / ubuntu:22.04 / fedora:latest containers on GitHub-hosted runners (runner size varies run-to-run: 72G vs 145G, differing CPU).
Screenshots / Logs
Representative (ubuntu:24.04, run 29608095905):
Failed to connect to ws://localhost:32929/task/service: connect ECONNREFUSED 127.0.0.1:32929
at TransportWebSocket._transportDisconnected (src/client/core/TransportWebSocket.ts:429)
● RocketRideClient Integration Tests › File Store Operations › should stat and delete a file
A worker process has failed to exit gracefully ... tests leaking due to improper teardown
Test Suites: 3 failed, 3 total
Affected runs:
What's needed to investigate (client / task-service side)
The crash angle is ruled out; the next step is data from a failing leg, not more C++ debugging:
- Engine-side log captured to a file, time-correlated with the jest run — to see whether the task service actually crashed/exited or simply never bound the dynamic port before the client dialed. (The engine logs to stdout by default; the test phase would need it redirected to a file per leg.)
- The jest run with
--detectOpenHandles (and ideally --runInBand) — to pin the leaked worker/handle and confirm whether the ECONNREFUSED is the cause or a symptom of teardown.
- The task-service lifecycle: which component allocates the dynamic port (e.g. 32929) and how the engine signals readiness to the client — is there a readiness/health check before the client dials, or a race where the client can connect before the port is bound (or after a per-task child exits)?
- A local reproduction loop: run the TS integration suite N× in a Linux container with CI-like CPU/memory limits, to reproduce off the matrix.
- Correlation with runner size/load: check whether failures cluster on the smaller/slower runners — a readiness timeout race that only trips under load.
Description
The
OS Matrix — compile + testworkflow intermittently fails its test phase on the Linux container legs. It is flaky: a leg that fails usually passes on re-run. The signature is the TypeScript client integration suite losing its WebSocket connection to the engine's task service (ECONNREFUSEDon the dynamic/task/serviceport), followed by a jest worker failing to exit.The C++ engine is not crashing. A minidump recovered during a failing run (ubuntu:24.04, run 29639419996) was symbolicated and turned out to be the deliberate crash-handling test's child process (
RR_CRASH_CHILD=crash, the intentional null-deref atengine-core/test/testMain.ipp:106). Crashpad catching that is the minidump-recovery test working as designed — it is a red herring, not the fault. This is also separate from the already-fixed stdout/uv-probe corruption (commit4c7b0c76).Steps to Reproduce
OS Matrix — compile + testonfeat/crashpad(Actions ▸ Run workflow).Unit testsstep.Expected vs Actual Behavior
Expected: the test phase passes deterministically for a given commit.
Actual: the TS client suite (
packages/client-typescript/tests/RocketRideClient.test.ts) intermittently cannot reach the engine's dynamic task-service WebSocket port —connect ECONNREFUSED— and a jest worker then fails to exit gracefully.Severity
P2 - Medium (feature impaired, workaround exists) — flaky; a re-run usually passes.
Platform
Version / Environment
feat/crashpad(PR #1524, breakpad→crashpad migration) · OS Matrix workflow · ubuntu:24.04 / ubuntu:22.04 / fedora:latest containers on GitHub-hosted runners (runner size varies run-to-run: 72G vs 145G, differing CPU).Screenshots / Logs
Representative (ubuntu:24.04, run 29608095905):
Affected runs:
What's needed to investigate (client / task-service side)
The crash angle is ruled out; the next step is data from a failing leg, not more C++ debugging:
--detectOpenHandles(and ideally--runInBand) — to pin the leaked worker/handle and confirm whether theECONNREFUSEDis the cause or a symptom of teardown.