fix: bound embedding page-read concurrency to avoid EMFILE on large vaults

[MarceloTrenkenchu]

Fixes #29.

Problem

loadPageContents in packages/engine/src/embeddings.ts fired one fs.readFile per vault page inside an unbounded Promise.all. On a vault with ~18k pages, swarmvault compile opens that many files at once and blows past the OS open-file-handle limit — EMFILE: too many open files on Windows, and the equivalent under ulimit -n elsewhere for large enough vaults. This reproduced on every compile run against our vault (18,423 pages), losing embedding coverage for a large, effectively random subset of pages each time.

The failure was also invisible from the CLI output: the .catch(() => { ...; return ""; }) discarded the caught error entirely, so EMFILE printed as an identical-looking "could not read page" warning to a genuinely missing or corrupt file. We only found the real cause by patching a local build to log err.code/err.message.

Full repro and root-cause writeup in #29.

Fix

• Added runWithConcurrency to packages/engine/src/utils.ts — this is the existing worker-pool helper that was already defined (unexported) inside orchestration.ts; moved it out and exported it instead of adding a second implementation, and updated orchestration.ts to import the shared one.
• loadPageContents now routes its page reads through runWithConcurrency with a concurrency cap of 64 instead of an unbounded Promise.all.
• The .catch handler now includes the caught error's code/message in the warning text, so a resource-exhaustion failure reads differently from a missing-file failure.

Testing

• Added packages/engine/test/page-read-concurrency.test.ts: covers runWithConcurrency's concurrency cap, result ordering under mixed completion timing, and edge cases (maxParallel below 1, maxParallel above task count).
• tsc --noEmit on packages/engine: clean.
• vitest run test/page-read-concurrency.test.ts test/retrieval.test.ts: all passing.
• Ran the full packages/engine suite both before and after this change on the same machine to separate signal from environment noise: 127/411 and 134/415 tests failed respectively — the +7 delta is exactly the 4 new tests (now passing) plus 3 extra file/test count from adding a file; the same pre-existing failures (missing built dist/hooks/*.js artifacts requiring a full pnpm build, and a few timeouts under slow disk I/O) appear in both runs. Nothing in vault.test.ts's failures touches embeddings.ts, orchestration.ts, or utils.ts.
• biome check on the changed files: no findings beyond CRLF line-ending noise from this being a fresh Windows checkout with core.autocrlf=true (confirmed the same noise appears on an untouched file, e.g. config.ts) — git diff itself is clean of line-ending churn.

Scope

Left the concurrency constant at 64 rather than making it configurable — no existing config surface distinguishes this from the embedding-batch size, and a fixed sane default is simplest. Happy to make it configurable via vault config if maintainers would prefer that.