biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Misc] refactor example - cpu_offload_lmcache (#17460)

Browse Source 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
This commit is contained in:
Reid
2025-05-01 23:05:24 +08:00
committed by GitHub
parent 460a2b1100
commit 7423cf0a9b
3 changed files with 63 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
examples/lmcache/README.md
View File

@@ -44,8 +44,8 @@ The main script generates several log files:
## 2. CPU Offload Examples
- `cpu_offload_lmcache_v0.py` - CPU offloading implementation for vLLM v0
- `cpu_offload_lmcache_v1.py` - CPU offloading implementation for vLLM v1
- `python cpu_offload_lmcache.py -v v0` - CPU offloading implementation for vLLM v0
- `python cpu_offload_lmcache.py -v v1` - CPU offloading implementation for vLLM v1
## 3. KV Cache Sharing