Skip to content

Improve the documentation of using vLLM. We now use vllm serve instead of using truss server.#472

Open
FredLiu876 wants to merge 18 commits into
mainfrom
fred/better-vllm-documentation
Open

Improve the documentation of using vLLM. We now use vllm serve instead of using truss server.#472
FredLiu876 wants to merge 18 commits into
mainfrom
fred/better-vllm-documentation

Conversation

@FredLiu876

Copy link
Copy Markdown
Contributor

https://linear.app/baseten/issue/FDE-776/update-vllm-example-in-truss-examples

  • Created two examples:
  1. Using vllm serve, recommended for 99% of users. OpenAI compatible.
  2. Existing example using truss server. No longer recommended, OpenAI compatible with slight modifications and use of Baseten bridge api. Only for custom inference.

@FredLiu876
FredLiu876 requested a review from dsingal0 July 25, 2025 19:35
@FredLiu876
FredLiu876 removed the request for review from dsingal0 July 25, 2025 19:54
@FredLiu876
FredLiu876 requested a review from dsingal0 July 25, 2025 21:36
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants