Make the concurrency harness run on Windows (portability) + document the RR-side caveat

[mithileshgau]

PR: Make the concurrency harness run on Windows (portability) + document the RR-side caveat

Branch: fix/windows-portability → main
Scope: OS-portability + honesty fixes only. No change to the benchmark methodology or the RocketRide-vs-LangChain numbers.

Why

Re-ran the full suite on Windows 11 (repo numbers are from a Mac), with the pinned engine (server-v3.2.1, the win64 build) and pinned langchain-core==0.3.86. Four of five headline results reproduced. Several harness-level assumptions broke before any measurement runs, and one RR-side result does not reproduce clean on Windows — documented openly rather than left for a re-runner to trip over.

What reproduced on Windows

• lines-of-code: 6.0× fewer lines (123 vs 689), 3.6× fewer chars. ✅
• authoring-effort: RR 0 imperative lines + validates; LC 14–17 lines, 5 hidden facts. ✅
• fault-isolation: RR isolation holds 10/10; in-process LangChain dies (exit 134, 0/4). ✅
• data-isolation: RR 0 lost / 0 leaked; LangChain silently drops 44–88% of 256. ✅
• concurrent-processing (LangChain side): .batch shared-conn crashes 64/64 (sqlite3.ProgrammingError); .abatch/seq serialize (~6.8s). ✅

Portability fixes (this PR)

• UTF-8 stdout/stderr (harness/__init__.py) — runners print unicode (→ × …); on the cp1252 Windows console a bare print() raises UnicodeEncodeError and aborts a run mid-benchmark (hit in data-isolation). No-op on POSIX.
• OS temp paths instead of /tmp (config.py, nodes/workload/IInstance.py, concurrent-processing/run.py) — /tmp doesn't exist on Windows. Harness and node defaults kept in sync; still overridable via $ROCKETRIDE_BENCH_PARAMS / $BENCH_DB_DIR. Verified out-of-the-box (data-isolation 0-lost with no env set).
• engine.exe on Windows (config.py) — so provenance()/engine_version() work instead of recording "unknown".
• provision.sh win64 branch — the pinned server-v3.2.1 release already ships win64.zip; the script just never wired it up. Also handles the venv Scripts/ layout.
• start_engine.sh — use engine.exe on Windows and actually forward --port=$PORT (it was computed for the health check but never passed to the engine).

Honesty fix

• concurrent-processing/run.py — the VERDICT console line hardcoded "ok" for the RR cell regardless of real status. results.json always recorded the true rr_topM_ok; the print now matches it (shows CHECK when node errors occur).

Known caveat this PR documents but does NOT fix (RR-side)

On Windows, concurrent-processing's RocketRide cell is status=check — ~8–12 of 72 docs raise sqlite3.ProgrammingError (thread-affinity). Process isolation still holds (M pipes = M PIDs), but the workload node caches a module-level sqlite connection, and the v3.2.1 engine dispatches a pipe's objects to different OS threads even at threads=1 (macOS/Linux use one thread per pipe). Only thread-affine resources are affected — data-isolation (GIL-safe list.append) reproduces 0-lost. The repo's own appendix honesty cell (1 pipe × threadCount=4 → 31 errors) already shows the mechanism; it just wasn't flagged as OS-dependent.

Fix options (team decision — see REPRODUCE-WINDOWS.md):

• A — node uses a thread-local connection → RR passes 0-errors everywhere (Mac numbers unchanged); keep the naive conn in the appendix. Concedes RR node authors need connection care too.
• B — keep naive-both; scope the "0 errors" row to macOS/Linux and report Windows degradation transparently; real fix lives in the engine.
• C — engine fix: make threads=1 mean one OS thread per pipe on Windows, restoring the naive-idiom-safety headline everywhere.

This PR is Option C-neutral: it only makes the suite run and report honestly on Windows, leaving A/B/C to the team.

Testing

All benches re-run on Windows 11 against the pinned v3.2.1 engine with no manual env vars (only ROCKETRIDE_URI for the non-default port): data-isolation 0-lost, fault-isolation 10/10, concurrent-processing runs and now reports the real RR status. No regressions.

🤖 Generated with Claude Code

[mithileshgau]

Data to settle the concurrent-processing Windows caveat

Ran the five checks against the committed run data (concurrent-work/runs-windows/) plus one control. Bottom line: same engine build as Mac, RR process isolation holds, and the only Windows failure is a thread-affine sqlite handle cached across the engine's thread-pool rotation — not a build mismatch, not the extension port, nothing deeper.

1. Engine build — build-mismatch refuted

Same logical build on both OSes (from provenance in every runs-windows/**/results.json):

	version	internal hash	build stamp	engine_sha256
Mac (darwin-arm64)	3.2.1.30	114509c6	2026-05-29T19:19:06Z	4ad67d11…
Windows (win64)	3.2.1.30	114509c6	2026-05-29T19:19:06Z	b844140a…

Identical version, internal hash, and build stamp. The engine_sha256 differs only because these are two platform compiles of that one build — the pinned public win64 release, not a dev build. So the behavior gap is same code, different platform, not a mismatched build.

2. Per-doc thread IDs — the smoking gun (already instrumented; tid= in every RRBENCH marker)

Distinct OS thread ids per pipe (pid), from the run-01 traces:

trace	pipes	max tids / pipe	pipes running docs on >1 thread
Mac rr.m8	8	1	0 / 8
Mac rr.m16	16	1	0 / 16
Windows rr.m8	8	2	2 / 8
Windows rr.m16	16	2	1 / 16

On Mac every pipe runs all its docs on one OS thread. On Windows some pipes rotate their (sequential) docs across two OS threads — confirming pool rotation under node dispatch even at threads=1. Reproduce by parsing tid=/pid= from runs-windows/concurrent-processing/run-01/trace/rr.m*.jsonl.gz.

4. Control — conn=per_call vs conn=module (same 64-doc workload, Windows :5566)

conn	node_errors	pipes rotating >1 tid
module (one cached connection)	2 (sqlite3.ProgrammingError)	2
per_call (fresh connection per doc)	0	3

Decisive: per_call saw more thread rotation (3 pipes) yet zero errors. yes-per_call / no-module → the failure is purely sqlite connection thread-affinity — a cached, thread-affine handle reused off-thread. No data race, no corruption; the per-doc work itself runs fine across threads. It's only the cached connection.

5. Extension engine on :5565 — ruled out

The harness connected to the freshly-started benchmark engine on :5566, not the extension's :5565:

• the orchestrator sets ROCKETRIDE_URI=ws://localhost:5566;
• the engine emitted 72 valid workload RRBENCH markers — only the benchmark bundle has that node; the extension's engine would reject use() with an unknown-provider error and produce none;
• results/engine.pid recorded the :5566 listener each start;
• provenance.engine_sha256 = b844140a… = the win64 3.2.1 binary we extracted.

3. For eng / CTO (data-informed; the invariant call is yours)

Empirically, on Windows threads=1 does not yield one OS thread per pipe — #2 shows pipes rotating across 2 tids, so there is a thread pool/executor under node dispatch that reuses/rotates worker threads; on Mac it's one thread per pipe. The open question is whether one-thread-per-pipe is an intended invariant (then Windows is an engine bug) or incidental (then benchmark/user nodes must never cache thread-affine handles). The control shows either fix closes it:

• A / node: thread-local or per-call connection in the workload node (the per_call run above is already clean, 0 errors);
• C / engine: pin one OS thread per pipe on Windows, restoring the naive-idiom-safety headline everywhere.

Net: same build; process isolation intact (M pipes = M PIDs); the Windows failure is a cached thread-affine sqlite connection meeting the engine's thread-pool rotation. Full evidence — 10× reps, gzipped traces, provenance — is in concurrent-work/runs-windows/.