Timeouts?

[boldandbusted]

Howdy. Is there some timeout that I can set (or disable) that will allow a model to complete an answer for a prompt?

I'm issuing commands with prompts like:

llm -m mistral-7b-instruct-v0 "Create a diagram using PlantUML that shows the relationships between HashiCorp Vault cluster members."

And, while I get output quickly, it prematurely ends mid-sentence. Apologies if this is in the documentation, but I did search and came up empty. Thanks for this awesome time-saving tool! :) Cheers!

[boldandbusted]

Sorry to bump this, but does no one else experience premature timeouts or cutoff answers?

[capi]

If I understand your question correctly, you hit the maximum output token limit of your LLM.
Most models I have experimented with, have a max_tokens option you can set.

So running it with llm -o max_tokens 4096 should increase the number of tokens you get back.

You can query the available options for your model with llm models --options.