biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,907 Commits 2 Branches 140 Tags
879a06579ea8a057c250e9b6cc4e632dabd87d2e
Commit Graph

73 Commits

Author SHA1 Message Date
Nicolò Lucchesi
0f95a1c3f2 [CI] Fix flaky test_two_responses_with_same_prev_id test (#27745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-29 15:10:35 +00:00
Russell Bryant
58fab50d82 [Frontend] Require flag for loading text and image embeds (#27204) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-22 15:52:02 +00:00
Hanchenli
7c572544e4 [GPT-OSS] Structure_Tag support for gpt-oss tool-call in cot (#25515) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
Signed-off-by: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Signed-off-by: Wei Wei <wwei6@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wei Wei <wwei6@meta.com>
Co-authored-by: Wei Wei <weiweinpu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-17 21:55:54 -07:00
Tahsin Tunan
43721bc67f [CI] Replace large models with tiny alternatives in tests (#24057) 
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-16 15:51:27 +01:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Cyrus Leung
dc7976dd9f [Misc] Upgrade more code to Python 3.10 (#26463) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 10:43:53 +01:00
Harry Mellor
2f99f2f506 Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Cyrus Leung
1e4ecca1d0 [V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Harry Mellor
6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Xiang Si
adae0c1f43 [CI/Build] do not enforce precompilation on tpu ci tests (#25992) 
Signed-off-by: Xiang Si <sixiang@google.com>
 2025-10-03 13:38:42 +00:00
Cyrus Leung
cd87bfbf37 [CI/Build] Reorganize root-level V1 tests (#25767) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-27 13:51:15 +08:00
Doug Smith
7ad5e50adf Improve output when failing json.loads() on structured output test (#25483) 
Signed-off-by: dougbtv <dosmith@redhat.com>
 2025-09-23 18:03:31 -06:00
Harry Mellor
875d6def90 Add backward compatibility for GuidedDecodingParams (#25422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-23 17:07:30 +01:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Zhuohan Li
6c47f6bfa4 [Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Flora Feng
15cb047e25 Extend renderer with embedding support and integrate completion endpoint (#24405) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 01:46:46 +08:00
22quinn
3a3e91bdfe [CI/Build] Disable flaky test_structured_output tests (#24404) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-08 02:51:59 +00:00
Aaron Pham
e67597545b [CI][Fix] deterministic seed for flaky CI runs on structured outputs (#24380) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-09-07 11:10:40 +08:00
Roger Wang
b121ca22ad [CI] Disable flaky structured output test from CI (#24366) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-06 13:31:56 +00:00
Kebe
8f423e5f43 [Feature][Response API] Add streaming support for non-harmony (#23741) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-04 17:49:06 +08:00
dsinghvi
70549c1245 [CI/Build] Serve images used by multimodal tests through local HTTP Server (#23907) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-03 16:13:11 +08:00
Noam Gat
39971db3aa Frontend: Adding LM Format Enforcer support to V1 engine (#22564) 
Signed-off-by: Noam Gat <noamgat@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-24 19:31:22 -07:00
Yong Hoon Shin
b6d7d34fc6 Add unit tests for batched guided and non-guided requests (#23389) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-22 10:31:24 -07:00
Cyrus Leung
8896eb72eb [Deprecation] Remove prompt_token_ids arg fallback in LLM.generate and LLM.embed (#18800) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-22 10:56:57 +08:00
Kebe
5368f76855 [Feature][Responses API] Support logprobs(non-stream) (#23319) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-08-21 23:09:16 +00:00
Michael Goin
93d0652433 [CI] Increase timeout for test_completion_with_image_embeds (#22670) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-11 20:31:36 -07:00
Chauncey
17eaaef595 [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-08-07 19:20:21 -07:00
Chen Zhang
4815b00f54 [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-07 08:33:25 -07:00
Michael Goin
a00d8b236f Use float32 for test_completion.py (#22385) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2025-08-07 11:07:47 +08:00
Woosuk Kwon
6d98843b31 [Responses API] Disable response store by default (#22137) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-03 04:04:21 -07:00
Nick Hill
316b1bf706 [Tests] Add tests for headless internal DP LB (#21450) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-23 07:49:25 -07:00
Chauncey
fdc5b43d20 [Bugfix]: Fix final_res_batch list index out of range error (#21055) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-17 00:29:09 -07:00
Chauncey
34cda778a0 [Frontend] OpenAI Responses API supports input image (#20975) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-15 18:59:36 -06:00
Nathan Hoos
d6902ce79f [V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975) 
Signed-off-by: Nathan Hoos <thwackyy.y@gmail.com>
 2025-07-10 15:30:26 -04:00
Woosuk Kwon
462b269280 Implement OpenAI Responses API [1/N] (#20504) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-06 18:32:13 -07:00
Nick Hill
67d25eca05 [Tests] Update online DP tests to verify that requests are balanced (#20157) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-03 14:49:13 +08:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Nick Hill
2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00
Feng XiaoLong
4fc1bf813a [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
Aaron Pham
2fc9075b82 [V1] Structured Outputs + Thinking compatibility (#16577) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-05-14 15:45:24 -07:00
Chauncey
dc1a821768 [Feature][V1] Support tool_choice: required when using Xgrammar as the StructuredOutputBackend. (#17845) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-05-12 23:01:31 -07:00
Russell Bryant
ebab1ac37c [CI] Make JSON output tests less likely to fail (#17859) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-05-12 22:31:54 +00:00
Cheng Kuan Yong Jason
08bf784078 [Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623) 
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-05-12 09:06:10 +08:00
Russell Bryant
6930a41116 [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-05-08 13:34:02 +08:00
Michael Goin
17b4d85f63 [CI][TPU] Skip structured outputs+spec decode tests on TPU (#17510) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-30 20:36:20 -07:00
Benjamin Chislett
34120f5acd [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-04-30 00:02:10 +00:00
Harry Mellor
a6977dbd15 Simplify (and fix) passing of guided decoding backend options (#17008) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-29 19:02:23 +00:00
Michał Moskal
86d9fc29cb implement Structural Tag with Guidance backend (#17333) 
Signed-off-by: Michal Moskal <michal@moskal.me>
 2025-04-29 02:21:32 +00:00
Russell Bryant
f8acd01ff7 [V1] Add structural_tag support using xgrammar (#17085) 2025-04-26 14:06:37 +00:00