biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Enforce no chunked prefill for embedding models (#10470)

Browse Source 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
This commit is contained in:
Cyrus Leung
2024-11-20 13:12:51 +08:00
committed by GitHub
parent ad44437ba3
commit b4be5a8adb
2 changed files with 60 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
docs/source/serving/compatibility_matrix.rst
View File

@@ -39,12 +39,13 @@ Feature x Feature
- :abbr:`prmpt adptr (Prompt Adapter)`
- :ref:`SD <spec_decode>`
- CUDA graph
- :abbr:`emd (Embedding Models)`
- :abbr:`enc-dec (Encoder-Decoder Models)`
- :abbr:`logP (Logprobs)`
- :abbr:`prmpt logP (Prompt Logprobs)`
- :abbr:`async output (Async Output Processing)`
- multi-step
- :abbr:`MM (Multimodal)`
- :abbr:`mm (Multimodal)`
- best-of
- beam-search
- :abbr:`guided dec (Guided Decoding)`
@@ -64,6 +65,7 @@ Feature x Feature
-
-
-
-
* - :ref:`APC <apc>`
- ✅
-
@@ -80,6 +82,7 @@ Feature x Feature
-
-
-
-
* - :ref:`LoRA <lora>`
- `✗ <https://github.com/vllm-project/vllm/pull/9057>`__
- ✅
@@ -96,6 +99,7 @@ Feature x Feature
-
-
-
-
* - :abbr:`prmpt adptr (Prompt Adapter)`
- ✅
- ✅
@@ -112,6 +116,7 @@ Feature x Feature
-
-
-
-
* - :ref:`SD <spec_decode>`
- ✗
- ✅
@@ -128,6 +133,7 @@ Feature x Feature
-
-
-
-
* - CUDA graph
- ✅
- ✅
@@ -144,6 +150,24 @@ Feature x Feature
-
-
-
-
* - :abbr:`emd (Embedding Models)`
- ✗
- ✗
- ✗
- ✗
- ✗
- ✗
-
-
-
-
-
-
-
-
-
-
* - :abbr:`enc-dec (Encoder-Decoder Models)`
- ✗
- `✗ <https://github.com/vllm-project/vllm/issues/7366>`__
@@ -151,6 +175,7 @@ Feature x Feature
- ✗
- `✗ <https://github.com/vllm-project/vllm/issues/7366>`__
- ✅
- ✅
-
-
-
@@ -166,7 +191,8 @@ Feature x Feature
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
- ✅
-
-
@@ -183,7 +209,8 @@ Feature x Feature
- ✅
- `✗ <https://github.com/vllm-project/vllm/pull/8199>`__
- ✅
- ✅
- ✗
- ✅
- ✅
-
-
@@ -199,6 +226,7 @@ Feature x Feature
- ✅
- ✗
- ✅
- ✗
- ✗
- ✅
- ✅
@@ -215,6 +243,7 @@ Feature x Feature
- ✅
- ✗
- ✅
- ✗
- ✗
- ✅
- `✗ <https://github.com/vllm-project/vllm/issues/8198>`__
@@ -224,14 +253,15 @@ Feature x Feature
-
-
-
* - :abbr:`MM (Multimodal)`
- `✗ <https://github.com/vllm-project/vllm/pull/8346>`__
* - :abbr:`mm (Multimodal)`
- ✅
- `✗ <https://github.com/vllm-project/vllm/pull/8348>`__
- `✗ <https://github.com/vllm-project/vllm/pull/7199>`__
- ?
- ?
- ✅
- ✗
- ✅
- ✅
- ✅
- ✅
- ✅
@@ -247,6 +277,7 @@ Feature x Feature
- ✅
- `✗ <https://github.com/vllm-project/vllm/issues/6137>`__
- ✅
- ✗
- ✅
- ✅
- ✅
@@ -263,6 +294,7 @@ Feature x Feature
- ✅
- `✗ <https://github.com/vllm-project/vllm/issues/6137>`__
- ✅
- ✗
- ✅
- ✅
- ✅
@@ -279,6 +311,7 @@ Feature x Feature
- ?
- ✅
- ✅
- ✗
- ?
- ✅
- ✅
@@ -353,6 +386,14 @@ Feature x Hardware
- ✅
- ✗
- ✅
* - :abbr:`emd (Embedding Models)`
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
* - :abbr:`enc-dec (Encoder-Decoder Models)`
- ✅
- ✅
@@ -361,6 +402,14 @@ Feature x Hardware
- ✅
- ✅
- ✗
* - :abbr:`mm (Multimodal)`
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - :abbr:`logP (Logprobs)`
- ✅
- ✅
@@ -393,14 +442,6 @@ Feature x Hardware
- ✅
- `✗ <https://github.com/vllm-project/vllm/issues/8477>`__
- ✅
* - :abbr:`MM (Multimodal)`
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - best-of
- ✅
- ✅