Add docstrings for LLMServer and related classes and examples (#142)

This commit is contained in:
Zhuohan Li
2023-06-07 18:25:20 +08:00
committed by GitHub
parent e38074b1e6
commit 4298374265
10 changed files with 212 additions and 18 deletions

View File

@@ -19,7 +19,7 @@ def main(args: argparse.Namespace):
SamplingParams(n=3, best_of=3, use_beam_search=True, temperature=0.0)),
]
# Run the server.
# Run the server by calling `server.step()` manually.
request_id = 0
while True:
# To test iteration-level scheduling, we add one request at each step.