biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194)

Browse Source 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
This commit is contained in:
Thomas Parnell
2025-07-19 21:27:21 +02:00
committed by GitHub
parent 9f414a12ad
commit 881e3cbe3b
10 changed files with 100 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
tests/models/language/generation/test_hybrid.py
View File

@@ -104,7 +104,6 @@ def test_models(
m.setenv("VLLM_ATTENTION_BACKEND", "FLASHINFER")
with vllm_runner(model,
max_num_seqs=MAX_NUM_SEQS,
enforce_eager=True,
enable_prefix_caching=False) as vllm_model:
vllm_v1_outputs = vllm_model.generate_greedy_logprobs(
example_prompts, max_tokens, num_logprobs)