biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
76 Commits 2 Branches 140 Tags
c84c708a1d13e5815376c2640cafeefaaf801827
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Woosuk Kwon c84c708a1d Add README
2023-02-24 12:04:49 +00:00
cacheflow
[Minor] Fix printing format
2023-02-24 11:56:06 +00:00
csrc
Add reshape_and_cache op
2023-02-18 19:22:57 +00:00
tests
Add tests for kernels
2023-02-18 19:23:07 +00:00
.gitignore
Add gitignore
2023-02-16 07:47:21 +00:00
README.md
Add README
2023-02-24 12:04:49 +00:00
server.py
Clean up the server script
2023-02-24 11:56:21 +00:00
setup.py
cache_kernel -> cache_kernels
2023-02-16 20:05:45 +00:00

README.md

CacheFlow

Installation

pip install cmake torch transformers
pip install -e .

Run

python server.py
Reference in New Issue View Git Blame Copy Permalink
Description
No description provided
Readme 180 MiB
Languages
Python 84.6%
Cuda 6%
JavaScript 4.1%
C++ 3.7%
Shell 0.9%
Other 0.6%