biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,213 Commits 2 Branches 140 Tags
58ce8d12b741b5bafe6bd0fb878727baea6171fe
Commit Graph

56 Commits

Author SHA1 Message Date
Harry Mellor
d9ab1ad9d1 reasoning_content -> reasoning (#27752) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-08 12:15:08 +00:00
Julien Denize
a404e2c0f1 Patch Mistral Tokenizer (#28146) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-11-06 06:43:16 +00:00
Julien Denize
73444b7b56 Performance fix MistralTokenizer: cache special ids and tokens (#27925) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-11-02 08:48:33 +00:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Julien Denize
c6187f55f7 Refactor MistralTokenizer (#26358) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-10-09 22:48:58 +00:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Bowen Bao
78b8015a4d [Bugfix] Relax tokenizer regex for mixtral to include 'tokenizer.model' (#25964) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2025-10-03 18:31:59 -04:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Zhuohan Li
6c47f6bfa4 [Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Jiangyun Zhu
885ca6d31d [Misc] Fix warnings for mistral model (#23552) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-08-29 06:58:48 +00:00
Jiangyun Zhu
625ccd1c4d [Bugfix] Replace custom Encoding class with BatchEncoding in MistralTokenizer (#22786) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-08-14 08:09:27 -07:00
Michard Hugo
25708d317a [Bugfix] Mistral crashes on tool with no description (#21167) 
Signed-off-by: HugoMichard <hugo@harfanglab.fr>
 2025-07-28 08:03:35 -07:00
Julien Denize
6d8d0a24c0 Add think chunk (#21333)
Some checks failed
Create Release / Create Release (push) Has been cancelled
Details 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-07-23 21:51:32 -07:00
Patrick von Platen
f20f9f063b [mistral_common] Add v11 tokenizer (#19193) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-06-05 08:27:41 -07:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Michael Goin
5873877241 [Bugfix] Mistral tool calling when content is list (#18729)
Some checks failed
Create Release / Create Release (push) Has been cancelled
Details 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-27 09:05:37 -07:00
Feng XiaoLong
4fc1bf813a [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
Harry Mellor
8c946cecca Update deprecated type hinting in vllm/transformers_utils (#18058) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 04:34:37 -07:00
Pete Savage
739e03b344 [Bugfix] Fixed mistral tokenizer path when pointing to file (#17457) 
Signed-off-by: Pete Savage <psavage@redhat.com>
 2025-04-30 08:08:37 -07:00
Gabriel Marinho
1c2bc7ead0 Truncation control for embedding models (#14776) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-04-30 09:24:57 +08:00
Cyrus Leung
93a126fbc7 [Misc] Make cached tokenizer pickle-compatible (#17048) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 13:05:00 +08:00
Guillaume Calmettes
1da6a09274 [Bugfix]: do not shutdown server if skip_special_use=False for MistralTokenizer (#14094) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-04-09 19:43:09 -07:00
Kebe
432cf22a6a [Bugfix] Fix regex compile display format (#15368) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-03-28 08:58:44 -07:00
Brayden Zhong
8b3e94a357 [Model] Remove duplicated message check in Mistral chat completion request (#15069) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-03-19 05:09:32 +00:00
Harry Mellor
3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-10 16:36:21 +00:00
Florian Greinacher
215bf150a6 [Bugfix] Handle None parameters in Mistral function calls. (#13786) 2025-02-26 03:06:21 -08:00
Rafael Vasquez
314cfade02 [Frontend] Generate valid tool call IDs when using tokenizer-mode=mistral (#12332) 2025-02-12 08:29:56 -08:00
Keyun Tong
3ee696a63d [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) 
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
 2025-02-12 12:25:58 +08:00
Florian Greinacher
cb080f32e3 [Bugfix] Support missing tool parameters in mistral tokenizer (#12884) 
Signed-off-by: Florian Greinacher <florian.greinacher@siemens.com>
 2025-02-11 03:33:33 +00:00
Patrick von Platen
d366ccc4e3 [RFC] [Mistral] FP8 format (#10130) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-02-08 14:12:53 -07:00
zifeitong
d01f66b039 [Bugfix] Fix multi-round chat error when mistral tokenizer is used (#12859) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-02-08 07:04:34 +00:00
youkaichao
20579c0fae make sure mistral_common not imported for non-mistral models (#12669) 
When people use deepseek models, they find that they need to solve cv2
version conflict, see https://zhuanlan.zhihu.com/p/21064432691 .

I added the check, and make all imports of `cv2` lazy.

---------

Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-03 13:40:25 +08:00
Russell Bryant
e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Kunshang Ji
54cacf008f [Bugfix] Mistral tokenizer encode accept list of str (#12149) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-01-17 16:47:53 +00:00
Wallas Henrique
8b79f9e107 [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) 2024-12-17 22:34:08 -08:00
Guillaume Calmettes
691a3ec047 [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (#10363) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-15 14:50:40 +00:00
Patrick von Platen
11cd1ae6ad [Tool parsing] Improve / correct mistral tool parsing (#10333) 2024-11-15 00:42:49 +00:00
Patrick von Platen
0535e5fe6c Fix edge case Mistral tokenizer (#10152) 2024-11-08 15:42:27 +00:00
Travis Johnson
2bcbae704c [Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-11-06 04:28:29 +00:00
Travis Johnson
1dd4cb2935 [Bugfix] Fix edge cases for MistralTokenizer (#9625) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2024-11-01 10:33:15 -07:00
Joe Runde
67bdf8e523 [Bugfix][Frontend] Guard against bad token ids (#9634) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-10-29 14:13:20 -07:00
sasha0552
337ed76671 [Bugfix] Fix offline mode when using mistral_common (#9457) 2024-10-18 18:12:32 -07:00
Cyrus Leung
1bbbcc0b1d [CI/Build] Fix lint errors in mistral tokenizer (#9504) 2024-10-19 00:09:35 +08:00
sasha0552
5e443b594f [Bugfix] Allow prefill of assistant response when using mistral_common (#9446) 2024-10-17 15:06:37 +00:00
Prashant Gupta
d11b46f3a5 [bugfix] fix f-string for error (#9295) 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
 2024-10-11 17:03:48 -07:00
Cyrus Leung
b0298aa8cc [Misc] Remove vLLM patch of BaichuanTokenizer (#8921) 2024-09-28 08:11:25 +00:00
Patrick von Platen
b4e4eda92e [Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640) 2024-09-20 14:33:03 -07:00
Patrick von Platen
a54ed80249 [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-09-17 17:50:37 +00:00
Cyrus Leung
7015417fd4 [Bugfix] Add missing attributes in mistral tokenizer (#8364) 2024-09-11 11:36:54 -07:00
Cyrus Leung
8c054b7a62 [Frontend] Clean up type annotations for mistral tokenizer (#8314) 2024-09-10 16:49:11 +00:00