mirror of https://github.com/vllm-project/vllm.git
486 B
486 B
title |
---|
BentoML |
{ #deployment-bentoml }
BentoML allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-compliant image and deploy it on Kubernetes.
For details, see the tutorial vLLM inference in the BentoML documentation.