biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Doc] Update docs for MM model development with context usage (#32691)

Browse Source 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
This commit is contained in:
Cyrus Leung
2026-01-21 02:37:35 +08:00
committed by GitHub
parent 9ab4388cd3
commit 09194b90a5
2 changed files with 20 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/online_serving/disaggregated_encoder/README.md
View File

@@ -38,7 +38,7 @@ Encoder engines should be launched with the following flags:
- `--max-num-batched-tokens=<large value>` **(default: 2048)** – This flag controls the token scheduling budget per decoding step and is irrelevant to encoder-only instances. **Set it to a very high value (effectively unlimited) to bypass scheduler limitations.** The actual token budget is managed by the encoder cache manager.
- `--mm-encoder-only` **(Optional)** - The language model is skipped during initialization to reduce device memory usage. **Models using this option must initialize the language component inside the context of `SupportsMultiModal._mark_language_model`.**
- `--mm-encoder-only` **(Optional)** - If possible, skips the language model during initialization to reduce device memory usage.
## Local media inputs