requirements.txt

cmake>=3.21
ninja  # For faster builds.
psutil
ray >= 2.9
sentencepiece  # Required for LLaMA tokenizer.
numpy
torch == 2.1.2
requests
py-cpuinfo
transformers >= 4.39.1  # Required for StarCoder2 & Llava.
xformers == 0.0.23.post1  # Required for CUDA 12.1.
fastapi
uvicorn[standard]
pydantic >= 2.0  # Required for OpenAI server.
prometheus_client >= 0.18.0
pynvml == 11.5.0
triton >= 2.1.0
outlines == 0.0.34
tiktoken == 0.6.0 # Required for DBRX tokenizer
Cmake based build system (#2830) 2024-03-18 18:38:33 -04:00			`cmake>=3.21`
Specify python package dependencies in requirements.txt (#78) 2023-05-07 16:30:43 -07:00			`ninja # For faster builds.`
			`psutil`
Update Ray version requirements (#2636) 2024-01-28 14:27:22 -08:00			`ray >= 2.9`
Specify python package dependencies in requirements.txt (#78) 2023-05-07 16:30:43 -07:00			`sentencepiece # Required for LLaMA tokenizer.`
			`numpy`
Pin PyTorch & xformers versions (#2155) 2023-12-17 01:46:54 -08:00			`torch == 2.1.2`
Usage Stats Collection (#2852) 2024-03-28 22:16:12 -07:00			`requests`
			`py-cpuinfo`
[Misc] Bump transformers version (#3592) 2024-03-24 06:32:45 -07:00			`transformers >= 4.39.1 # Required for StarCoder2 & Llava.`
[Minor] Fix xformers version (#2158) 2023-12-17 02:28:02 -08:00			`xformers == 0.0.23.post1 # Required for CUDA 12.1.`
Specify python package dependencies in requirements.txt (#78) 2023-05-07 16:30:43 -07:00			`fastapi`
Use standard extras for uvicorn (#1166) 2023-09-27 21:41:36 -03:00			`uvicorn[standard]`
migrate pydantic from v1 to v2 (#2531) 2024-01-22 01:05:56 +01:00			`pydantic >= 2.0 # Required for OpenAI server.`
Restrict prometheus_client >= 0.18.0 to prevent errors when importing pkgs (#3070) 2024-02-28 13:38:26 +08:00			`prometheus_client >= 0.18.0`
Implement custom all reduce kernels (#2192) 2024-01-28 04:46:35 +08:00			`pynvml == 11.5.0`
Require triton >= 2.1.0 (#2746) Co-authored-by: yangrui1 <yangrui@lanjingren.com> 2024-02-05 15:07:36 +08:00			`triton >= 2.1.0`
[Hotfix] [Debug] test_openai_server.py::test_guided_regex_completion (#3383) 2024-03-13 17:02:21 -07:00			`outlines == 0.0.34`
[Model] Add support for DBRX (#3660) 2024-03-27 13:01:46 -07:00			`tiktoken == 0.6.0 # Required for DBRX tokenizer`