[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355)

Signed-off-by: jiang1.li <jiang1.li@intel.com>
2024-11-20 18:57:39 +08:00
parent d5b28447e0
commit 63f1fde277
8 changed files with 558 additions and 368 deletions
--- a/docs/source/serving/compatibility_matrix.rst
+++ b/docs/source/serving/compatibility_matrix.rst
@@ -344,7 +344,7 @@ Feature x Hardware
     - ✅
     - ✅
     - ✅
-     - ✗ 
+     - ✅
     - ✅
   * - :ref:`APC <apc>`
     - `✗ <https://github.com/vllm-project/vllm/issues/3687>`__ 
@@ -352,7 +352,7 @@ Feature x Hardware
     - ✅
     - ✅
     - ✅
-     - ✗
+     - ✅
     - ✅
   * - :ref:`LoRA <lora>`
     - ✅