vLLM
/
vllm-project.github.io
mirror of https://github.com/vllm-project/vllm-project.github.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove invalid links for references 

Ugh for some reason the links from internal notion from our side was still there, my bad.
This commit is contained in:
Aaron Pham 2025-01-14 19:34:03 -05:00 committed by GitHub
parent f9a15b52eb
commit 793b30ceac
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_posts/2025-01-14-struct-decode-intro.md
View File

@ -109,7 +109,7 @@ There are still a few usability concerns in XGrammar v0 integration to match fea
With the release of [v1](https://github.com/vllm-project/vllm/issues/8779) on the horizon, we're working on a tentative plan for structured decoding:
1. Moving guided decoding towards scheduler-level [\[10\]](https://www.notion.so/Blog-4X-structured-decoding-speed-in-vLLM-8c3f2d44f6504202abbdb534983f2b2e?pvs=21)
1. Moving guided decoding towards scheduler-level:
- Reason: We have more context regarding which requests that use structured decoding at a scheduler-level, therefore it shouldn't block other requests within the batch (tentatively addressing **limitation (2)**). In a sense, this moves guided decoding outside of the critical path.
- This would allow for more natural vertical integration with jump-forward decoding (address **limitation (4)**).
2. Allowing bit-mask calculation in one process instead of each GPU workers