vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[Doc] Fix code formatting in spec_decode.rst (#9348)

This commit is contained in:
Michael Goin 2024-10-15 00:29:11 -04:00 committed by GitHub
parent 44eaa5a5d9
commit 8e836d982a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/models/spec_decode.rst
View File

@ -44,10 +44,10 @@ To perform the same with an online mode launch the server:
.. code-block:: bash .. code-block:: bash
python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 8000 --model facebook/opt-6.7b \ python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 8000 --model facebook/opt-6.7b \
--seed 42 -tp 1 --speculative_model facebook/opt-125m --use-v2-block-manager \ --seed 42 -tp 1 --speculative_model facebook/opt-125m --use-v2-block-manager \
--num_speculative_tokens 5 --gpu_memory_utilization 0.8 --num_speculative_tokens 5 --gpu_memory_utilization 0.8
Then use a client: Then use a client:
.. code-block:: python .. code-block:: python