Add memory analyzer & utomatically configure KV cache size (#6)

2023-03-11 23:23:14 -08:00
parent 1a7eb7da61
commit e9d3f2ff77
7 changed files with 216 additions and 34 deletions
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
 ## Installation

 ```bash
-pip install cmake torch transformers
+pip install psutil numpy torch transformers
 pip install flash-attn # This may take up to 10 mins.
 pip install -e .
 ```