biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,309 Commits 2 Branches 140 Tags
0e9358c11daf3f5a2d4e8f80a100b6d5e070e1a1
Commit Graph

82 Commits

Author SHA1 Message Date
Juan Pérez de Algaba
b111f8a61f fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit (#37952) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-03-27 09:02:10 -04:00
Kunshang Ji
53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Russell Bryant
d460a18fc6 [Docs] Expand --allowed-media-domains security guidance with threat details (#36506) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-03-09 17:43:42 +00:00
Harry Mellor
a0f44bb616 Allow markdownlint to run locally (#36398) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:05:24 -07:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Russell Bryant
636ee223ac [Docs] Document security risks of GPT-OSS Python tool (#35139) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-03-04 20:27:31 +00:00
Manrique Vargas
28028dff2f fix(docs): use static rdzv backend in multi-node troubleshooting script (#34784) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-03-04 17:15:35 +00:00
Seungmin Kim
160424a937 [Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs (#33992) 
Signed-off-by: Seungmin Kim <8457324+ehfd@users.noreply.github.com>
Signed-off-by: Andrew Mello <19512127+88plug@users.noreply.github.com>
Co-authored-by: 88plug <19512127+88plug@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 18:15:51 -08:00
Mark McLoughlin
5cc7c4452e [Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950) 
Export the existing Model FLOPs Utilization (MFU) metrics via Prometheus.

`--enable-mfu-metrics` is required for these to be exposed.

Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-02-23 15:01:07 +00:00
Isotr0py
19fab44152 [Doc] Update Encoder-Decoder models support doc with Florence-2 (#34581) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 04:18:57 -08:00
Nicolò Lucchesi
81a90e5277 [Docs] Add bart-plugin to docs (#33905) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-05 12:20:25 +00:00
Wentao Ye
d88a1df699 [Deprecation] Deprecate profiling envs (#33722) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-04 05:58:21 +00:00
jma99_2333
22d9a056d5 Support clear mm and encoder cache (#33452) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-31 15:22:25 +00:00
Andrew Bennett
f243abc92d Fix various typos found in docs (#32212) 
Signed-off-by: Andrew Bennett <potatosaladx@meta.com>
 2026-01-13 03:41:47 +00:00
Harry Mellor
0b544e6476 [Docs] Fix some snippets (#31378) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-26 12:47:41 +00:00
Isotr0py
2532f437ee [Doc] Add troubleshooting for Triton PTX error about undefined gpu-name (#31338) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-25 02:26:34 -08:00
Didier Durand
1a55cfafcb [Doc]: fixing typos in various files (#30540) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Didier Durand <2927957+didier-durand@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-14 02:14:37 -08:00
Mark McLoughlin
2dcbac9077 [Docs] Generate full list of metrics in user docs (#30388) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-10 16:09:34 +00:00
Russell Bryant
b08025a83b [Docs] Discuss api key limitations in security guide (#29922) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-12-02 20:57:28 -08:00
Cyrus Leung
389aa1b2eb [Doc] Update more docs with respect to V1 (#29188) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-23 10:58:48 +08:00
Cyrus Leung
aab0102a26 [V0 deprecation] Remove more V0 references (#29088) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:56:59 +00:00
Cyrus Leung
56e96b37e4 [V0 Deprecation] Remove best_of (#29090) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:40:40 +08:00
Cyrus Leung
89d3679221 [Doc] Fix failing doc build (#28772) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-15 05:33:27 -08:00
youkaichao
155ad56d7b [doc] add guide about the provided PTX was compiled with an unsupported toolchain (#28305) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-11-08 00:26:34 +08:00
wangxiyuan
428bc7bf1c [V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Harry Mellor
483ea64611 [Docs] Replace all explicit anchors with real links (#27087) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:22:06 -07:00
cong-meta
bbc1b29665 Update troubleshooting.md and remind VLLM_TRACE_FUNCTION usage (#27069) 
Signed-off-by: cong-meta <prowindy@hotmail.com>
 2025-10-17 01:53:06 -07:00
Harry Mellor
4ffd6e8942 [Docs] Reduce custom syntax used in docs (#27009) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-16 20:05:34 -07:00
Cyrus Leung
9c4cb68339 [Chore] Remove SupportsV0Only interface and update supported models docs (#26783) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-14 04:55:10 -07:00
Chenheli Hua
ad87ba927a [Small] Prevent bypassing media domain restriction via HTTP redirects (#26035) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-10-02 10:27:10 -07:00
Russell Bryant
3958b96bf5 Add option to restrict media domains (#25783) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-27 01:23:52 +00:00
Harry Mellor
8c853050e7 [Docs] Enable fail_on_warning for the docs build in CI (#25580) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-24 19:30:33 +00:00
Woosuk Kwon
759ef49b15 Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Russell Bryant
ba6011027d [Docs] Update V1 doc to reflect whisper support (#24606) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-11 01:50:08 -07:00
youkaichao
8c5a747246 [distributed] update known issues (#24624) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-11 11:09:38 +08:00
Mickaël Seznec
ed16d0f26f [Doc] mention fpdb for multiprocess breakpoints (#24452) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-09-08 21:46:45 -07:00
nopperl
fa4311d85f [V1] v1 engine + full CUDA graph support for PLaMo2 (#23998) 
Signed-off-by: Hemmi Shinichi <shemmi@preferred.jp>
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
Co-authored-by: Hemmi Shinichi <shemmi@preferred.jp>
Co-authored-by: Thomas Parnell <tom.parnell@gmail.com>
 2025-09-03 08:24:02 -07:00
youkaichao
f38035c123 [distributed][rl] remove nccl cumem env var override (#24141) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-03 06:45:25 +00:00
Thomas Parnell
1c26b42296 [Docs] [V1] [Hybrid] Add new documentation re: contributing mamba-based models (#23824) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-29 18:47:58 +00:00
Didier Durand
d99c3a4f7b [Doc]: fix typos in .md files (including those of #23751) (#23825) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-28 04:38:19 -07:00
Thomas Parnell
704432af3c [V1] [Hybrid] Disable prefix caching by default for hybrid or mamba-based models (#23716) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-27 12:51:54 +00:00
Thomas Parnell
227e231b55 [Docs] [V1] [Hybrid] Update docs to remove FlashInfer constraint for hybrid models (#23665) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-26 18:33:16 +00:00
Chen Zhang
a073be6d87 [Doc] Update the doc for log probs + prefix caching (#23399) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-22 13:20:39 +00:00
22quinn
f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
Asaf Joseph Gardin
3663870c72 [V1][Mamba1] - Full CUDA and Piecewise CUDA Graphs Support (#23035) 
Signed-off-by: asafg <asafg@ai21.com>
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-08-20 20:08:51 -07:00
Csrayz
a0632a3e03 [Frontend] Expose do_log_stats interval to env (#22905) 
Signed-off-by: Csrayz <jover@cmbchina.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-15 13:00:20 +00:00
Harry Mellor
767e63b860 [Docs] Improve docs navigation (#22720) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-12 04:25:55 -07:00
Chendi.Xue
59f3b93636 [DOC] update v1_guide with INTEL HW (#22679) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-08-12 01:22:49 -07:00
Eric Curtin
3fa5b25845 Document aarch64 CPU support works (#22646) 
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
 2025-08-11 07:22:45 -07:00
Thomas Parnell
2be07a0db1 Update docs for Minimax-Text support (#22562) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-09 00:18:18 -07:00