[Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824)

Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
This commit is contained in:
billishyahao
2025-03-20 15:00:33 +08:00
committed by GitHub
parent bfe2fe0af4
commit 742369d35a
3 changed files with 70 additions and 21 deletions

View File

@@ -589,6 +589,7 @@ class DeepseekV2Model(nn.Module):
model_config = vllm_config.model_config
cache_config = vllm_config.cache_config
quant_config = vllm_config.quant_config
self.config = config
self.vocab_size = config.vocab_size