results(linux-x64): epyc7b13 8v 10x run — isolation ratios reproduce vs Mac baseline#7
Open
EinarMontalvo wants to merge 1 commit into
Conversation
…vs Mac baseline Runtime 3.2.1.30 (114509c6) x86_64, langchain-core 0.3.86. All 4 groups 10/10 clean: concurrent-processing (LangChain .batch crashes 0/64), fault-isolation (LC loses all 0/4), data-isolation (LC silently loses 2-95%), authoring-effort.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📊 Summary
Linux-x64 (AMD EPYC 7B13, 8 vCPU / QEMU) 10× run of the isolation/concurrency harness — the diff vs main is the linux numbers side-by-side with the Mac baseline (and alongside
results/linux-x64-c7i-20260709).Runtime 3.2.1.30 (114509c6) x86_64 · langchain-core 0.3.86. All four groups 10/10 reps clean, and the isolation ratios reproduce cross-platform (absolute timings differ from Mac/c7i, as expected):
.batch(shared conn) crashes 0/64.abatchloses all 0/4Ran natively via the real C++
./engine ai/eaas.py(not the bundled python — that would void the isolation measurement).🛠️ Provisioning notes (Ubuntu 24.04) — not fixed in this PR
On a fresh Ubuntu 24.04 host, the documented one-click path still blocked before the engine could start. These host/setup gaps are not fixed in this PR (results-only):
pythonmissing — Ubuntu shipspython3only; some docs/commands expectpython. Workaround:apt install python-is-python3(or always usepython3/ the venv binary).libc++.so.1missing — Linuxenginelinks against LLVM libc++. Withoutlibc++1/libc++1-18, startup fails with:error while loading shared libraries: libc++.so.1: cannot open shared object filescripts/setup-ubuntu.shdoes not currently install it.After installing those manually,
make start+ fullrun_isolated.shcompleted cleanly.scripts/start_engine.shreadiness gate can false-pass — same issue already mentioned in PR #3; still not fixed here.On connection-refused,
curl -w '%{http_code}'prints000and|| echo 000appends another →"000000", which!= "000"→ script declares the engine healthy before it is listening. We hit this on Ubuntu 24.04 when the binary failed to loadlibc++:✅ Test plan
python-is-python3,libc++1)make provision/make provision-competitorsmake start→ real healthy status (HTTP 401)REPS=10 bash run_isolated.sh→ all 4 groups 10/10 cleanpython aggregate.py→ scorecard refreshed fromruns/