biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Doc] Convert docs to use colon fences (#12471)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor
2025-01-29 03:38:29 +00:00
committed by GitHub
parent a7e3eba66f
commit dd6a3a02cb
68 changed files with 2352 additions and 2341 deletions
Download Patch File Download Diff File Expand all files Collapse all files

879
docs/source/features/compatibility_matrix.md
View File

@@ -4,13 +4,13 @@
The tables below show mutually exclusive features and the support on some hardware.
```{note}
:::{note}
Check the '✗' with links to see tracking issue for unsupported feature/hardware combination.
```
:::
## Feature x Feature
```{raw} html
:::{raw} html
<style>
/* Make smaller to try to improve readability */
td {
@@ -23,448 +23,447 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
font-size: 0.8rem;
}
</style>
```
:::
```{list-table}
:header-rows: 1
:stub-columns: 1
:widths: auto
:::{list-table}
:header-rows: 1
:stub-columns: 1
:widths: auto
* - Feature
- [CP](#chunked-prefill)
- [APC](#automatic-prefix-caching)
- [LoRA](#lora-adapter)
- <abbr title="Prompt Adapter">prmpt adptr</abbr>
- [SD](#spec_decode)
- CUDA graph
- <abbr title="Pooling Models">pooling</abbr>
- <abbr title="Encoder-Decoder Models">enc-dec</abbr>
- <abbr title="Logprobs">logP</abbr>
- <abbr title="Prompt Logprobs">prmpt logP</abbr>
- <abbr title="Async Output Processing">async output</abbr>
- multi-step
- <abbr title="Multimodal Inputs">mm</abbr>
- best-of
- beam-search
- <abbr title="Guided Decoding">guided dec</abbr>
* - [CP](#chunked-prefill)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
* - [APC](#automatic-prefix-caching)
- ✅
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
* - [LoRA](#lora-adapter)
- [✗](gh-pr:9057)
- ✅
-
-
-
-
-
-
-
-
-
-
-
-
-
-
* - <abbr title="Prompt Adapter">prmpt adptr</abbr>
- ✅
- ✅
- ✅
-
-
-
-
-
-
-
-
-
-
-
-
-
* - [SD](#spec_decode)
- ✅
- ✅
- ✗
- ✅
-
-
-
-
-
-
-
-
-
-
-
-
* - CUDA graph
- ✅
- ✅
- ✅
- ✅
- ✅
-
-
-
-
-
-
-
-
-
-
-
* - <abbr title="Pooling Models">pooling</abbr>
- ✗
- ✗
- ✗
- ✗
- ✗
- ✗
-
-
-
-
-
-
-
-
-
-
* - <abbr title="Encoder-Decoder Models">enc-dec</abbr>
- ✗
- [✗](gh-issue:7366)
- ✗
- ✗
- [✗](gh-issue:7366)
- ✅
- ✅
-
-
-
-
-
-
-
-
-
* - <abbr title="Logprobs">logP</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
- ✅
-
-
-
-
-
-
-
-
* - <abbr title="Prompt Logprobs">prmpt logP</abbr>
- ✅
- ✅
- ✅
- ✅
- [✗](gh-pr:8199)
- ✅
- ✗
- ✅
- ✅
-
-
-
-
-
-
-
* - <abbr title="Async Output Processing">async output</abbr>
- ✅
- ✅
- ✅
- ✅
- ✗
- ✅
- ✗
- ✗
- ✅
- ✅
-
-
-
-
-
-
* - multi-step
- ✗
- ✅
- ✗
- ✅
- ✗
- ✅
- ✗
- ✗
- ✅
- [✗](gh-issue:8198)
- ✅
-
-
-
-
-
* - <abbr title="Multimodal Inputs">mm</abbr>
- ✅
- [✗](gh-pr:8348)
- [✗](gh-pr:7199)
- ?
- ?
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ?
-
-
-
-
* - best-of
- ✅
- ✅
- ✅
- ✅
- [✗](gh-issue:6137)
- ✅
- ✗
- ✅
- ✅
- ✅
- ?
- [✗](gh-issue:7968)
- ✅
-
-
-
* - beam-search
- ✅
- ✅
- ✅
- ✅
- [✗](gh-issue:6137)
- ✅
- ✗
- ✅
- ✅
- ✅
- ?
- [✗](gh-issue:7968>)
- ?
- ✅
-
-
* - <abbr title="Guided Decoding">guided dec</abbr>
- ✅
- ✅
- ?
- ?
- [✗](gh-issue:11484)
- ✅
- ✗
- ?
- ✅
- ✅
- ✅
- [✗](gh-issue:9893)
- ?
- ✅
- ✅
-
```
- * Feature
* [CP](#chunked-prefill)
* [APC](#automatic-prefix-caching)
* [LoRA](#lora-adapter)
* <abbr title="Prompt Adapter">prmpt adptr</abbr>
* [SD](#spec_decode)
* CUDA graph
* <abbr title="Pooling Models">pooling</abbr>
* <abbr title="Encoder-Decoder Models">enc-dec</abbr>
* <abbr title="Logprobs">logP</abbr>
* <abbr title="Prompt Logprobs">prmpt logP</abbr>
* <abbr title="Async Output Processing">async output</abbr>
* multi-step
* <abbr title="Multimodal Inputs">mm</abbr>
* best-of
* beam-search
* <abbr title="Guided Decoding">guided dec</abbr>
- * [CP](#chunked-prefill)
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
- * [APC](#automatic-prefix-caching)
* ✅
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
- * [LoRA](#lora-adapter)
* [✗](gh-pr:9057)
* ✅
*
*
*
*
*
*
*
*
*
*
*
*
*
*
- * <abbr title="Prompt Adapter">prmpt adptr</abbr>
* ✅
* ✅
* ✅
*
*
*
*
*
*
*
*
*
*
*
*
*
- * [SD](#spec_decode)
* ✅
* ✅
* ✗
* ✅
*
*
*
*
*
*
*
*
*
*
*
*
- * CUDA graph
* ✅
* ✅
* ✅
* ✅
* ✅
*
*
*
*
*
*
*
*
*
*
*
- * <abbr title="Pooling Models">pooling</abbr>
* ✗
* ✗
* ✗
* ✗
* ✗
* ✗
*
*
*
*
*
*
*
*
*
*
- * <abbr title="Encoder-Decoder Models">enc-dec</abbr>
* ✗
* [✗](gh-issue:7366)
* ✗
* ✗
* [✗](gh-issue:7366)
* ✅
* ✅
*
*
*
*
*
*
*
*
*
- * <abbr title="Logprobs">logP</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✗
* ✅
*
*
*
*
*
*
*
*
- * <abbr title="Prompt Logprobs">prmpt logP</abbr>
* ✅
* ✅
* ✅
* ✅
* [✗](gh-pr:8199)
* ✅
* ✗
* ✅
* ✅
*
*
*
*
*
*
*
- * <abbr title="Async Output Processing">async output</abbr>
* ✅
* ✅
* ✅
* ✅
* ✗
* ✅
* ✗
* ✗
* ✅
* ✅
*
*
*
*
*
*
- * multi-step
* ✗
* ✅
* ✗
* ✅
* ✗
* ✅
* ✗
* ✗
* ✅
* [✗](gh-issue:8198)
* ✅
*
*
*
*
*
- * <abbr title="Multimodal Inputs">mm</abbr>
* ✅
* [✗](gh-pr:8348)
* [✗](gh-pr:7199)
* ?
* ?
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ?
*
*
*
*
- * best-of
* ✅
* ✅
* ✅
* ✅
* [✗](gh-issue:6137)
* ✅
* ✗
* ✅
* ✅
* ✅
* ?
* [✗](gh-issue:7968)
* ✅
*
*
*
- * beam-search
* ✅
* ✅
* ✅
* ✅
* [✗](gh-issue:6137)
* ✅
* ✗
* ✅
* ✅
* ✅
* ?
* [✗](gh-issue:7968>)
* ?
* ✅
*
*
- * <abbr title="Guided Decoding">guided dec</abbr>
* ✅
* ✅
* ?
* ?
* [✗](gh-issue:11484)
* ✅
* ✗
* ?
* ✅
* ✅
* ✅
* [✗](gh-issue:9893)
* ?
* ✅
* ✅
*
:::
(feature-x-hardware)=
## Feature x Hardware
```{list-table}
:header-rows: 1
:stub-columns: 1
:widths: auto
:::{list-table}
:header-rows: 1
:stub-columns: 1
:widths: auto
* - Feature
- Volta
- Turing
- Ampere
- Ada
- Hopper
- CPU
- AMD
* - [CP](#chunked-prefill)
- [✗](gh-issue:2729)
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - [APC](#automatic-prefix-caching)
- [✗](gh-issue:3687)
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - [LoRA](#lora-adapter)
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - <abbr title="Prompt Adapter">prmpt adptr</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- [✗](gh-issue:8475)
- ✅
* - [SD](#spec_decode)
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - CUDA graph
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
- ✅
* - <abbr title="Pooling Models">pooling</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ?
* - <abbr title="Encoder-Decoder Models">enc-dec</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
* - <abbr title="Multimodal Inputs">mm</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - <abbr title="Logprobs">logP</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - <abbr title="Prompt Logprobs">prmpt logP</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - <abbr title="Async Output Processing">async output</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✗
- ✗
* - multi-step
- ✅
- ✅
- ✅
- ✅
- ✅
- [✗](gh-issue:8477)
- ✅
* - best-of
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - beam-search
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
* - <abbr title="Guided Decoding">guided dec</abbr>
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
- ✅
```
- * Feature
* Volta
* Turing
* Ampere
* Ada
* Hopper
* CPU
* AMD
- * [CP](#chunked-prefill)
* [✗](gh-issue:2729)
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * [APC](#automatic-prefix-caching)
* [✗](gh-issue:3687)
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * [LoRA](#lora-adapter)
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * <abbr title="Prompt Adapter">prmpt adptr</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* [✗](gh-issue:8475)
* ✅
- * [SD](#spec_decode)
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * CUDA graph
* ✅
* ✅
* ✅
* ✅
* ✅
* ✗
* ✅
- * <abbr title="Pooling Models">pooling</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ?
- * <abbr title="Encoder-Decoder Models">enc-dec</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✗
- * <abbr title="Multimodal Inputs">mm</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * <abbr title="Logprobs">logP</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * <abbr title="Prompt Logprobs">prmpt logP</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * <abbr title="Async Output Processing">async output</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✗
* ✗
- * multi-step
* ✅
* ✅
* ✅
* ✅
* ✅
* [✗](gh-issue:8477)
* ✅
- * best-of
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * beam-search
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
- * <abbr title="Guided Decoding">guided dec</abbr>
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
:::