biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Fix handling of max_num_batched_tokens for pooling tasks (#23004)

Browse Source 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
This commit is contained in:
Maximilien de Bayser
2025-08-16 14:36:30 -03:00
committed by GitHub
parent 829bbd7882
commit 52ce1420e9
2 changed files with 5 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/config/__init__.py
View File

@@ -3600,9 +3600,6 @@ class VllmConfig:
logger.info(reason)
self.scheduler_config.chunked_prefill_enabled = False
self.scheduler_config.long_prefill_token_threshold = 0
self.scheduler_config.max_num_batched_tokens = max(
self.scheduler_config.max_model_len,
DEFAULT_MAX_NUM_BATCHED_TOKENS)
if self.cache_config is not None:
self.cache_config.enable_prefix_caching = False