Fix concurrent-processing on Windows (per-thread SQLite connection) + Windows portability#2
Merged
Merged
Conversation
…Windows portability The workload node cached a module-level SQLite connection (conn="module"); the win64 engine dispatches a pipe's sequential docs across >1 OS thread even at threads=1 (macOS/Linux use one thread per pipe), so the cached connection was reused off-thread -> sqlite3.ProgrammingError on ~9-12/72 docs. The headline cell now uses a per-thread cached connection (conn="thread_local") — safe under either thread topology and identical on macOS (one thread per pipe). The rr_appendix_threads4 honesty cell keeps the naive shared connection (conn="module") to still demonstrate the thread-affinity trap. Parity gate is unaffected (it compares _sqlite_doc_work, which takes the connection as an argument). Also includes the Windows portability fixes from PR #1: UTF-8 stdout, OS temp paths, engine.exe provenance, provision.sh win64 branch, start_engine.sh port forwarding, and the VERDICT-print honesty fix (reflects the real rr_topM_ok). REPRODUCE-WINDOWS.md is a Windows setup guide. Co-authored-by: mithileshgau <mithileshgau@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
✅ Validated on Windows 11 — the fix works. Full 10× re-run (engine
Honest caveat (not an error): the M=16 cell is Full evidence + two runner fixes (the runner was missing a node-install step, so |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes the one Windows-specific failure #1 diagnosed —
concurrent-processing's RocketRide cell raisingsqlite3.ProgrammingErroron Windows — and folds in that PR's Windows portability fixes (credit to @mithileshgau).Root cause (proven in #1)
The synthetic workload node cached a module-level SQLite connection (
conn="module"). SQLite connections are thread-affine, and thewin64server-v3.2.1engine dispatches a pipe's sequential docs across >1 OS thread even atthreads=1(macOS/Linux use one thread per pipe), so the cached connection was reused off-thread →ProgrammingErroron ~9–12/72 docs. RocketRide process isolation held everywhere (M pipes = M PIDs);data-isolation/fault-isolationreproduced clean. This is not an engine bug —threads=1doesn't promise one OS thread per pipe.The fix (node)
The headline cell now uses a per-thread cached connection (
conn="thread_local"innodes/workload/IInstance.py) — safe under either thread topology, and identical on macOS (one thread per pipe → one connection per pipe, exactly as before). Therr_appendix_threads4honesty cell keeps the naive shared connection (conn="module") so the thread-affinity trap is still demonstrated. The parity gate is unaffected — it compares_sqlite_doc_work(which takes the connection as an argument); connection acquisition is outside it.Mechanism proof (deterministic, any OS): a shared connection created on one live thread and used from another raises the exact Windows
ProgrammingError; a thread-local connection is clean.Portability (from #1, credited)
UTF-8 stdout, OS temp paths instead of
/tmp,engine.exeprovenance,provision.shwin64 branch,start_engine.shport forwarding, and the VERDICT-print honesty fix (the console line now reflects the realrr_topM_okinstead of a hardcoded"ok"—results.jsonwas always correct).REPRODUCE-WINDOWS.mdis a Windows setup guide.Deliberately not in this PR
runs-windows/) — to be added after a clean Windows run on this branch (concurrent-processing RR should now be 0 errors, 5/5).Thanks @mithileshgau for the Windows run and the airtight diagnosis. 🙌