biondizzle/vllm-with-lmcache
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: biondizzle:master
Branches Tags
biondizzle:master biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:glm51-cu130
...
pull from: biondizzle:mla-multi-group-kv-cache
Branches Tags
biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:master biondizzle:glm51-cu130

1 Commits
master ... mla-multi-

Author SHA1 Message Date
Matthew Harris
e52c6f0a98 patch to fix MLA multi-group KV cache 2026-05-05 21:10:05 -04:00
1 changed files with 2 additions and 2 deletions
Expand all files Collapse all files

4
Dockerfile
View File

@@ -13,9 +13,9 @@ RUN apt-get update && apt-get install -y git \
libcurand-dev-13-0 \ libcurand-dev-13-0 \
libcufft-dev-13-0 \ libcufft-dev-13-0 \
libnvjitlink-dev-13-0 && \ libnvjitlink-dev-13-0 && \
git clone https://github.com/biondizzle/LMCache.git /tmp/lmcache && \ git clone https://github.com/Byteflux/LMCache.git /tmp/lmcache && \
cd /tmp/lmcache && \ cd /tmp/lmcache && \
git checkout feat/redis-ttl && \ git checkout mla-multi-group-kv-cache-with-redis && \
CUDA_HOME=/usr/local/cuda \ CUDA_HOME=/usr/local/cuda \
TORCH_CUDA_ARCH_LIST="10.0" \ TORCH_CUDA_ARCH_LIST="10.0" \
pip install --no-cache-dir --no-build-isolation . && \ pip install --no-cache-dir --no-build-isolation . && \