mirror of https://github.com/vllm-project/vllm.git
[Doc] Fix code formatting in spec_decode.rst (#9348)
This commit is contained in:
parent
44eaa5a5d9
commit
8e836d982a
|
@ -44,10 +44,10 @@ To perform the same with an online mode launch the server:
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 8000 --model facebook/opt-6.7b \
|
python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 8000 --model facebook/opt-6.7b \
|
||||||
--seed 42 -tp 1 --speculative_model facebook/opt-125m --use-v2-block-manager \
|
--seed 42 -tp 1 --speculative_model facebook/opt-125m --use-v2-block-manager \
|
||||||
--num_speculative_tokens 5 --gpu_memory_utilization 0.8
|
--num_speculative_tokens 5 --gpu_memory_utilization 0.8
|
||||||
|
|
||||||
Then use a client:
|
Then use a client:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue