I tried llmwiki on my company's documentation repo with over 900 markdown files generating ~33,000 chunks. The compilation was taking ages because each page and chunk was embedded with a separate API call, and on large wikis the JSON embedding store would eventually hit Node's string-length ceiling.
I implemented two opt-in features behind env vars to address this:
- LLMWIKI_BATCH_EMBEDDINGS=true — groups texts into single API calls instead of one-by-one
- LLMWIKI_BINARY_EMBEDDINGS=true — stores vectors in a compact Float32 .bin file instead of JSON
Both default to off, so the existing behavior is unchanged. With both enabled, compilation became orders of magnitude faster, and query/view/context all worked as expected on a 900+ page wiki.
I opened PR #98 but I figured it would be useful to have an issue too. I am sorry if I am stepping ahead of myself by opening the PR without discussing the issue, but I wasn't planning on merging this at first because I was just testing. But since it worked, I decided it would be worth bringing it here.
Thank you for your time!
I tried llmwiki on my company's documentation repo with over 900 markdown files generating ~33,000 chunks. The compilation was taking ages because each page and chunk was embedded with a separate API call, and on large wikis the JSON embedding store would eventually hit Node's string-length ceiling.
I implemented two opt-in features behind env vars to address this:
Both default to off, so the existing behavior is unchanged. With both enabled, compilation became orders of magnitude faster, and query/view/context all worked as expected on a 900+ page wiki.
I opened PR #98 but I figured it would be useful to have an issue too. I am sorry if I am stepping ahead of myself by opening the PR without discussing the issue, but I wasn't planning on merging this at first because I was just testing. But since it worked, I decided it would be worth bringing it here.
Thank you for your time!