Name		Name	Last commit message	Last commit date
configs		configs
utils		utils
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
__init__.py		__init__.py
requirements.txt		requirements.txt
serve.py		serve.py

Repository files navigation

LLM Serving using Customerized APIs

Customerized APIs for serving LLMs locally.

Deployment

uvicorn --host 0.0.0.0 serve:app

About

Basic APIs for serving LLMs locally.

llama2-13b llama2-7b llm-serving

0 stars

1 watching

Report repository

Releases

Packages

Used by

Contributors

Languages