biondizzle/vllm-with-lmcache
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: biondizzle:dream-build
Branches Tags
biondizzle:master biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:glm51-cu130
...
pull from: biondizzle:mla-multi-group-kv-cache
Branches Tags
biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:master biondizzle:glm51-cu130

2 Commits
dream-buil ... mla-multi-

Author SHA1 Message Date
Matthew Harris
e52c6f0a98 patch to fix MLA multi-group KV cache 2026-05-05 21:10:05 -04:00
biondizzle
82330a31b1 add await to redis write 2026-04-17 07:19:15 +00:00
1 changed files with 3 additions and 3 deletions
Expand all files Collapse all files

6
Dockerfile
View File

@@ -13,13 +13,13 @@ RUN apt-get update && apt-get install -y git \
libcurand-dev-13-0 \
libcufft-dev-13-0 \
libnvjitlink-dev-13-0 && \
git clone https://github.com/biondizzle/LMCache.git /tmp/lmcache && \
git clone https://github.com/Byteflux/LMCache.git /tmp/lmcache && \
cd /tmp/lmcache && \
git checkout feat/redis-ttl && \
git checkout mla-multi-group-kv-cache-with-redis && \
CUDA_HOME=/usr/local/cuda \
TORCH_CUDA_ARCH_LIST="10.0" \
pip install --no-cache-dir --no-build-isolation . && \
rm -rf /tmp/lmcache
rm -rf /tmp/lmcache && export CACHE_BUSTER=1
# Copy over nemotron reasonong parser
COPY ./super_v3_reasoning_parser.py /opt/super_v3_reasoning_parser.py