From 41cd6ebf99f7efc4dbdb2c9164b7a8a9ac50c838 Mon Sep 17 00:00:00 2001
From: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Date: Fri, 24 Jan 2025 11:24:43 -0600
Subject: [PATCH 1/6] Add files via upload
---
_posts/2025-01-21-stack-release (1).md | 106 +++++++++++++++++++++++++
1 file changed, 106 insertions(+)
create mode 100644 _posts/2025-01-21-stack-release (1).md
diff --git a/_posts/2025-01-21-stack-release (1).md b/_posts/2025-01-21-stack-release (1).md
new file mode 100644
index 0000000..f6d1a51
--- /dev/null
+++ b/_posts/2025-01-21-stack-release (1).md
@@ -0,0 +1,106 @@
+---
+layout: post
+title: "High Performance and Easy Deployment of vLLM in K8S with “vLLM production-stack”"
+thumbnail-img: /assets/img/stack-thumbnail.png
+share-img: /assets/img/stack-thumbnail.png
+author: LMCache Team
+image: /assets/img/stack-thumbnail.png
+---
+
+
+
+## TL;DR
+- **vLLM** boasts the largest open-source community, but what does it take to transform vLLM from the best single-node LLM engine to a premier LLM serving system?
+- **Today, we release “vLLM production-stack”**, a vLLM-based full inference stack that introduces two major advantages:
+ - **10x better performance** (3-10x lower response delay & 2-5x higher throughput) with prefix-aware request routing and KV-cache sharing.
+ - **Easy cluster deployment** with built-in support for fault tolerance, autoscaling, and observability.
+- And the best part? It’s **open-source**—so everyone can get started right away! [[**https://github.com/vllm-project/production-stack**]](https://github.com/vllm-project/production-stack)
+
+
+# The Context
+
+
+*In the AI arms race, it’s no longer just about who has the best model—it’s about **who has the best LLM serving system**.*
+
+**vLLM** has taken the open-source community by storm, with unparalleled hardware and model support plus an active ecosystem of top-notch contributors. But until now, vLLM has mostly focused on **single-node** deployments.
+
+How do we extend its power into a **full-stack** inference system that any organization can deploy at scale with *high reliability*, *high throughput*, and *low latency*? That’s precisely why the LMCache team and the vLLM team built **vLLM production-stack**.
+
+
+
+
+
+
+
+
+
+
+
+
+
+