[Frontend][Core] Add shutdown timeout - allowing in-flight requests to finish (#34730)
Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -327,6 +327,12 @@ class VllmConfig:
|
||||
weight_transfer_config: WeightTransferConfig | None = None
|
||||
"""The configurations for weight transfer during RL training."""
|
||||
|
||||
shutdown_timeout: int = Field(default=0, ge=0)
|
||||
"""Shutdown grace period for in-flight requests. Shutdown will be delayed for
|
||||
up to this amount of time to allow already-running requests to complete. Any
|
||||
remaining requests are aborted once the timeout is reached.
|
||||
"""
|
||||
|
||||
def compute_hash(self) -> str:
|
||||
"""
|
||||
WARNING: Whenever a new field is added to this config,
|
||||
|
||||
Reference in New Issue
Block a user