vllm/docs/configuration/serve_args.md

title
Server Arguments

{ #serve-args }

The vllm serve command is used to launch the OpenAI-compatible server.

CLI Arguments

The vllm serve command is used to launch the OpenAI-compatible server. To see the available CLI arguments, run vllm serve --help!

Configuration file

You can load CLI arguments via a YAML config file. The argument names must be the long form of those outlined [above][serve-args].

For example:

# config.yaml

model: meta-llama/Llama-3.1-8B-Instruct
host: "127.0.0.1"
port: 6379
uvicorn-log-level: "info"

To use the above config file:

vllm serve --config config.yaml

!!! note In case an argument is supplied simultaneously using command line and the config file, the value from the command line will take precedence. The order of priorities is command line > config file values > defaults. e.g. vllm serve SOME_MODEL --config config.yaml, SOME_MODEL takes precedence over model in config file.