[doc] Fold long code blocks to improve readability (#19926)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
This commit is contained in:
@@ -60,22 +60,22 @@ And then you can send out a query to the OpenAI-compatible API to check the avai
|
||||
curl -o- http://localhost:30080/models
|
||||
```
|
||||
|
||||
Expected output:
|
||||
??? Output
|
||||
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
```json
|
||||
{
|
||||
"id": "facebook/opt-125m",
|
||||
"object": "model",
|
||||
"created": 1737428424,
|
||||
"owned_by": "vllm",
|
||||
"root": null
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "facebook/opt-125m",
|
||||
"object": "model",
|
||||
"created": 1737428424,
|
||||
"owned_by": "vllm",
|
||||
"root": null
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
To send an actual chatting request, you can issue a curl request to the OpenAI `/completion` endpoint:
|
||||
|
||||
@@ -89,23 +89,23 @@ curl -X POST http://localhost:30080/completions \
|
||||
}'
|
||||
```
|
||||
|
||||
Expected output:
|
||||
??? Output
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "completion-id",
|
||||
"object": "text_completion",
|
||||
"created": 1737428424,
|
||||
"model": "facebook/opt-125m",
|
||||
"choices": [
|
||||
```json
|
||||
{
|
||||
"text": " there was a brave knight who...",
|
||||
"index": 0,
|
||||
"finish_reason": "length"
|
||||
"id": "completion-id",
|
||||
"object": "text_completion",
|
||||
"created": 1737428424,
|
||||
"model": "facebook/opt-125m",
|
||||
"choices": [
|
||||
{
|
||||
"text": " there was a brave knight who...",
|
||||
"index": 0,
|
||||
"finish_reason": "length"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
### Uninstall
|
||||
|
||||
@@ -121,23 +121,25 @@ sudo helm uninstall vllm
|
||||
|
||||
The core vLLM production stack configuration is managed with YAML. Here is the example configuration used in the installation above:
|
||||
|
||||
```yaml
|
||||
servingEngineSpec:
|
||||
runtimeClassName: ""
|
||||
modelSpec:
|
||||
- name: "opt125m"
|
||||
repository: "vllm/vllm-openai"
|
||||
tag: "latest"
|
||||
modelURL: "facebook/opt-125m"
|
||||
??? Yaml
|
||||
|
||||
replicaCount: 1
|
||||
```yaml
|
||||
servingEngineSpec:
|
||||
runtimeClassName: ""
|
||||
modelSpec:
|
||||
- name: "opt125m"
|
||||
repository: "vllm/vllm-openai"
|
||||
tag: "latest"
|
||||
modelURL: "facebook/opt-125m"
|
||||
|
||||
requestCPU: 6
|
||||
requestMemory: "16Gi"
|
||||
requestGPU: 1
|
||||
replicaCount: 1
|
||||
|
||||
pvcStorage: "10Gi"
|
||||
```
|
||||
requestCPU: 6
|
||||
requestMemory: "16Gi"
|
||||
requestGPU: 1
|
||||
|
||||
pvcStorage: "10Gi"
|
||||
```
|
||||
|
||||
In this YAML configuration:
|
||||
* **`modelSpec`** includes:
|
||||
|
||||
Reference in New Issue
Block a user