biondizzle
45b6109ee1
- Switch to official facebook/xformers (johnnynunez fork has TORCH_STABLE_ONLY requiring PyTorch headers not in 2.9.0) - Increase MAX_JOBS from 2-4 to 8 for all builds (native GH200 has 97GB HBM3) - Increase NVCC_THREADS from 1 to 4 for flash-attention
VLLM images for GH200
Hosted here
docker login
# Alternative
# docker buildx build --platform linux/arm64 --memory=600g -t rajesh550/gh200-vllm:0.9.0.1 .
docker build --memory=450g --platform linux/arm64 -t rajesh550/gh200-vllm:0.11.1rc2 . 2>&1 | tee build.log
docker push rajesh550/gh200-vllm:0.11.1rc2