biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Fix Sparse24 Compressed Tensors models (#33446)

Browse Source 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
This commit is contained in:
Kyle Sayers
2026-02-12 02:15:16 -05:00
committed by GitHub
parent 80f2ba6ea6
commit e9cd691132
3 changed files with 17 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
csrc/sparse/cutlass/sparse_scaled_mm_entry.cu
View File

@@ -6,11 +6,11 @@
#include "cutlass_extensions/common.hpp"
bool cutlass_sparse_scaled_mm_supported(int64_t cuda_device_capability) {
// sparse CUTLASS kernels need at least
// sparse CUTLASS kernels need exactly hopper and are not forward compatible
// CUDA 12.2 and SM90 (Hopper)
#if defined CUDA_VERSION
return CUDA_VERSION >= 12020 && cuda_device_capability >= 90;
return CUDA_VERSION >= 12020 && cuda_device_capability == 90;
#endif
return false;
@@ -98,7 +98,7 @@ std::vector<torch::Tensor> cutlass_sparse_compress(torch::Tensor const& a) {
TORCH_CHECK_NOT_IMPLEMENTED(
false,
"No compiled cutlass_sparse_compress for a compute capability less than "
"No compiled cutlass_sparse_compress for a compute capability equal to "
"CUDA device capability: ",
version_num);
}