[Docs] Update readme (#7316)
This commit is contained in:
@@ -31,8 +31,10 @@ vLLM is fast with:
|
||||
* Efficient management of attention key and value memory with **PagedAttention**
|
||||
* Continuous batching of incoming requests
|
||||
* Fast model execution with CUDA/HIP graph
|
||||
* Quantization: `GPTQ <https://arxiv.org/abs/2210.17323>`_, `AWQ <https://arxiv.org/abs/2306.00978>`_, `SqueezeLLM <https://arxiv.org/abs/2306.07629>`_, FP8 KV Cache
|
||||
* Optimized CUDA kernels
|
||||
* Quantization: `GPTQ <https://arxiv.org/abs/2210.17323>`_, `AWQ <https://arxiv.org/abs/2306.00978>`_, INT4, INT8, and FP8
|
||||
* Optimized CUDA kernels, including integration with FlashAttention and FlashInfer.
|
||||
* Speculative decoding
|
||||
* Chunked prefill
|
||||
|
||||
vLLM is flexible and easy to use with:
|
||||
|
||||
@@ -41,9 +43,9 @@ vLLM is flexible and easy to use with:
|
||||
* Tensor parallelism and pipeline parallelism support for distributed inference
|
||||
* Streaming outputs
|
||||
* OpenAI-compatible API server
|
||||
* Support NVIDIA GPUs and AMD GPUs
|
||||
* (Experimental) Prefix caching support
|
||||
* (Experimental) Multi-lora support
|
||||
* Support NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs and GPUs, PowerPC CPUs, TPU, and AWS Neuron.
|
||||
* Prefix caching support
|
||||
* Multi-lora support
|
||||
|
||||
For more information, check out the following:
|
||||
|
||||
@@ -53,7 +55,6 @@ For more information, check out the following:
|
||||
* :ref:`vLLM Meetups <meetups>`.
|
||||
|
||||
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user