vllm/benchmarks/kernels at 799b5721f6bb89ad03e47ba79131e0ce65023635 - vllm

cpu

[cpu][bench] Add CPU paged attention benchmarks (#31720 )

2026-01-06 10:57:57 +00:00

deepgemm

[Doc]: fix typos in various files (#29230 )

2025-11-24 11:10:48 +00:00

bench_block_fp8_gemm.py

Remove weight_scale.T special case for SM90 Block FP8 CUTLASS kernel (#28431 )

2025-11-11 11:46:04 -05:00

bench_fp8_gemm.py

[Misc] Add SPDX-FileCopyrightText (#20428 )

2025-07-04 07:40:42 +00:00

bench_int8_gemm.py

[Benchmark] Refactor benchmark script for fp8 & int8 (#19627 )

2025-06-15 15:15:37 +08:00

bench_mxfp4_qutlass.py

[Transform] [Quantization] Add QuTLASS support to vLLM (#24440 )

2025-10-10 09:43:40 -07:00

bench_nvfp4_gemm.py

Enable Fbgemm NVFP4 on Dense models (#25609 )

2025-09-24 21:12:53 -07:00

bench_nvfp4_quant.py

[NVFP4][Perf] Tune NVFP4 input quant kernel for small batch size (#30897 )

2025-12-21 09:41:57 -08:00

bench_nvfp4_qutlass.py

[Transform] [Quantization] Add QuTLASS support to vLLM (#24440 )

2025-10-10 09:43:40 -07:00

bench_per_token_quant_fp8.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_2d_silu_mul_fp8_quant.py

[Performance][DP/EP] Add silu_mul_per_token_group_quant_fp8_colmajor kernel (#29470 )

2025-12-03 18:04:59 +00:00

benchmark_activation.py

[Perf][Kernels] Vectorize csrc/activations_kernels.cu (#29512 )

2025-12-16 14:56:02 -08:00

benchmark_bitblas.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_cutlass_moe_fp8.py

[Bugfix] Add init_workspace_manager to moe kernel benchmarks (#31042 )

2026-01-05 19:14:33 -08:00

benchmark_cutlass_moe_nvfp4.py

[Bugfix] Add init_workspace_manager to moe kernel benchmarks (#31042 )

2026-01-05 19:14:33 -08:00

benchmark_device_communicators.py

use the same stream for cuda graph catpure and replay for NCCL (#29207 )

2025-12-25 19:10:03 +08:00

benchmark_fused_collective.py

[PERF] Allreduce fusion. Support torch native matching. Tuning of the thresholds (#24248 )

2025-11-10 18:33:11 -05:00

benchmark_grouped_gemm_cutlass.py

[Bugfix] Add init_workspace_manager to moe kernel benchmarks (#31042 )

2026-01-05 19:14:33 -08:00

benchmark_layernorm.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_lora.py

Load tuned fused_moe_lora shrink and expand kernel configs separately (#27435 )

2025-11-04 18:27:35 +08:00

benchmark_machete.py

[Kernel][Quantization] add w4a8 support for marlin kernel (#24722 )

2025-11-29 07:19:33 -08:00

benchmark_marlin.py

[Kernel][Quantization] add w4a8 support for marlin kernel (#24722 )

2025-11-29 07:19:33 -08:00

benchmark_mla_k_concat.py

[perf] Use direct copy (broadcast) instead of cat for k_nope/k_pe in MLA prefill (#29710 )

2025-12-11 08:20:45 +00:00

benchmark_moe_align_block_size.py

[Kernel][MoE] optimize moe_align_block_size (#29642 )

2025-12-07 01:58:47 -08:00

benchmark_moe_permute_unpermute.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

benchmark_moe.py

[Benchmark] Fix OOM during MoE kernel tuning for large models (#31604 )

2026-01-02 22:24:51 +00:00

benchmark_mrope.py

Standardise get_rope to use rope_parameters["partial_rotary_factor"], not rotary_dim (#30389 )

2025-12-11 20:45:23 +00:00

benchmark_paged_attention.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_per_token_group_quant.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

benchmark_quant.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_reshape_and_cache_flash.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_reshape_and_cache.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

benchmark_rmsnorm.py

Update Optional[x] -> x | None and Union[x, y] to x | y (#26633 )

2025-10-12 09:51:31 -07:00

benchmark_rope.py

Standardise get_rope to use rope_parameters["partial_rotary_factor"], not rotary_dim (#30389 )

2025-12-11 20:45:23 +00:00

benchmark_shapes.py

Disable nm-testing models with issues in CI (#28206 )

2025-11-06 06:19:07 -08:00

benchmark_silu_mul_fp8_quant.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

benchmark_trtllm_decode_attention.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

benchmark_trtllm_prefill_attention.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

benchmark_w8a8_block_fp8.py

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985 )

2025-11-18 11:34:36 -08:00

graph_machete_bench.py

[Chore]:Extract math and argparse utilities to separate modules (#27188 )

2025-10-26 04:03:32 -07:00

requirements.txt

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701 )

2024-09-23 13:46:26 -04:00

utils.py

Update Optional[x] -> x | None and Union[x, y] to x | y (#26633 )

2025-10-12 09:51:31 -07:00

weight_shapes.py

[kernel] Support W4A8 on Hopper (#23198 )

2025-08-24 06:18:04 +00:00