[Core] Shut down aDAG workers with clean async llm engine exit (#7224)

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
This commit is contained in:
Rui Qiao
2024-08-12 17:57:16 -07:00
committed by GitHub
parent 774cd1d3bf
commit 198d6a2898
5 changed files with 40 additions and 25 deletions

View File

@@ -36,6 +36,7 @@ class AsyncEngineRPCServer:
"""Cleanup all resources."""
self.socket.close()
self.context.destroy()
self.engine.shutdown_background_loop()
async def get_model_config(self, identity):
"""Send the ModelConfig"""