biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Hardware][Intel GPU] Add Intel GPU(XPU) inference backend (#3814)

Browse Source 
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Abhilash Majumder <abhilash.majumder@intel.com>
Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
This commit is contained in:
Kunshang Ji
2024-06-18 02:01:25 +08:00
committed by GitHub
parent 1f12122b17
commit 728c4c8a06
31 changed files with 1998 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/model_executor/layers/vocab_parallel_embedding.py
View File

@@ -307,7 +307,7 @@ class VocabParallelEmbedding(torch.nn.Module):
else:
masked_input = input_
# Get the embeddings.
output_parallel = F.embedding(masked_input, self.weight)
output_parallel = F.embedding(masked_input.long(), self.weight)
# Mask the output embedding.
if self.tp_size > 1:
output_parallel.masked_fill_(input_mask.unsqueeze(1), 0)