biondizzle/vllm-with-lmcache
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: biondizzle:dream-build-glm-save-decode
Branches Tags
biondizzle:master biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:glm51-cu130
...
pull from: biondizzle:dream-build
Branches Tags
biondizzle:dream-build-glm biondizzle:dream-build biondizzle:rocm-mla-multi-group-kv-cache biondizzle:dream-build-glm-save-decode biondizzle:mla-multi-group-kv-cache biondizzle:glm51-cu130-nightly biondizzle:openai-rocm-nightly biondizzle:deepseekv4-cu130 biondizzle:custom-weights-cu130 biondizzle:master biondizzle:glm51-cu130

1 Commits
dream-buil ... dream-buil

Author SHA1 Message Date
biondizzle
c0c05d5572 official tweax 2026-05-07 19:10:31 +00:00
2 changed files with 2 additions and 2 deletions
Expand all files Collapse all files

2
Dockerfile
View File

@@ -18,7 +18,7 @@ RUN apt-get update && apt-get install -y git \
CUDA_HOME=/usr/local/cuda \
TORCH_CUDA_ARCH_LIST="10.0" \
pip install --no-cache-dir --no-build-isolation . && \
rm -rf /tmp/lmcache && export CACHE_BUSTER=3
rm -rf /tmp/lmcache && export CACHE_BUSTER=4
# Make sure we have the latest up to date chat template
COPY glm_5.1_chat_template.jinja /opt/chat_template.jinja

2
lmcache-config-glm-51.yaml
View File

@@ -1,7 +1,7 @@
chunk_size: 256
local_cpu: true
max_local_cpu_size: 256.0
save_decode_cache: false
save_decode_cache: true
enable_lazy_memory_allocator: true
lazy_memory_initial_ratio: 1.0
use_gpu_connector_v3: true