biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[CI/Build] Add markdown linter (#11857)

Browse Source 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
This commit is contained in:
Rafael Vasquez
2025-01-12 03:17:13 -05:00
committed by GitHub
parent b25cfab9a0
commit 43f3d9e699
49 changed files with 585 additions and 560 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/source/deployment/frameworks/cerebrium.md
View File

@@ -13,14 +13,14 @@ vLLM can be run on a cloud based GPU machine with [Cerebrium](https://www.cerebr
To install the Cerebrium client, run:
```console
$ pip install cerebrium
$ cerebrium login
pip install cerebrium
cerebrium login
```
Next, create your Cerebrium project, run:
```console
$ cerebrium init vllm-project
cerebrium init vllm-project
```
Next, to install the required packages, add the following to your cerebrium.toml:
@@ -58,10 +58,10 @@ def run(prompts: list[str], temperature: float = 0.8, top_p: float = 0.95):
Then, run the following code to deploy it to the cloud:
```console
$ cerebrium deploy
cerebrium deploy
```
If successful, you should be returned a CURL command that you can call inference against. Just remember to end the url with the function name you are calling (in our case` /run`)
If successful, you should be returned a CURL command that you can call inference against. Just remember to end the url with the function name you are calling (in our case`/run`)
```python
curl -X POST https://api.cortex.cerebrium.ai/v4/p-xxxxxx/vllm/run \

10
docs/source/deployment/frameworks/dstack.md
View File

@@ -13,16 +13,16 @@ vLLM can be run on a cloud based GPU machine with [dstack](https://dstack.ai/),
To install dstack client, run:
```console
$ pip install "dstack[all]
$ dstack server
pip install "dstack[all]
dstack server
```
Next, to configure your dstack project, run:
```console
$ mkdir -p vllm-dstack
$ cd vllm-dstack
$ dstack init
mkdir -p vllm-dstack
cd vllm-dstack
dstack init
```
Next, to provision a VM instance with LLM of your choice (`NousResearch/Llama-2-7b-chat-hf` for this example), create the following `serve.dstack.yml` file for the dstack `Service`:

12
docs/source/deployment/frameworks/skypilot.md
View File

@@ -334,12 +334,12 @@ run: |
1. Start the chat web UI:
```console
sky launch -c gui ./gui.yaml --env ENDPOINT=$(sky serve status --endpoint vllm)
```
```console
sky launch -c gui ./gui.yaml --env ENDPOINT=$(sky serve status --endpoint vllm)
```
2. Then, we can access the GUI at the returned gradio link:
```console
| INFO | stdout | Running on public URL: https://6141e84201ce0bb4ed.gradio.live
```
```console
| INFO | stdout | Running on public URL: https://6141e84201ce0bb4ed.gradio.live
```