[FEATURE] llama-server --rpc #928
arturbacilla
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Llama-server has an optional feature to share memory with multiple GPU and multiple hosts (using the flag --rpc ). But since this is a experimental feature, it is not provided within the docker image but only by building it. I was wondering if there is a way to llama-swap to have a experimental image with this --rpc flag available, or if one could replace the binary inside the container somehow
All reactions