Vadim Gimpelson 58eee5f2e0 [PERF] Use faster way of decode in tokenizer: avoid useless list-to-list conversion (#20000) ...

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>

2025-08-02 01:43:52 -07:00

..

chat_templates

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

configs

[Speculative Decoding] Add speculators config support (#21345)

2025-08-01 08:25:18 -04:00

processors

[Misc] Remove redundant config definitions (#21891)

2025-07-29 23:54:18 -07:00

tokenizers

[Bugfix] Mistral crashes on tool with no description (#21167)

2025-07-28 08:03:35 -07:00

__init__.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

config.py

[Speculative Decoding] Add speculators config support (#21345)

2025-08-01 08:25:18 -04:00

detokenizer_utils.py

[Bugfix]: Fix messy code when using logprobs (#20910)

2025-07-14 11:06:45 +00:00

detokenizer.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

dynamic_module.py

[Deprecation][2/N] Replace --task with --runner and --convert (#21470)

2025-07-27 19:42:40 -07:00

processor.py

[Misc] Automatically resolve HF processor init kwargs (#22005)

2025-07-31 22:44:10 -07:00

s3_utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tokenizer_base.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tokenizer_group.py

[Deprecation][2/N] Replace --task with --runner and --convert (#21470)

2025-07-27 19:42:40 -07:00

tokenizer.py

[PERF] Use faster way of decode in tokenizer: avoid useless list-to-list conversion (#20000)

2025-08-02 01:43:52 -07:00

utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00