biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618)

Browse Source 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
This commit is contained in:
Rui Qiao
2024-08-16 20:44:05 -07:00
committed by GitHub
parent 6bd19551b0
commit bae888cb8e
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/entrypoints/openai/rpc/server.py
View File

@@ -38,6 +38,8 @@ class AsyncEngineRPCServer:
self.socket.close() self.socket.close()
self.context.destroy() self.context.destroy()
self.engine.shutdown_background_loop() self.engine.shutdown_background_loop()
# Clear the engine reference so that it can be GC'ed.
self.engine = None
async def get_model_config(self, identity): async def get_model_config(self, identity):
"""Send the ModelConfig""" """Send the ModelConfig"""