biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Doc] fix doc string typo in block_manager swap_out function (#10212)

Browse Source
This commit is contained in:
Yangcheng Li
2024-11-12 00:53:07 +08:00
committed by GitHub
parent 874f551b36
commit 36fc439de0
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/core/block_manager.py
View File

@@ -393,7 +393,7 @@ class SelfAttnBlockSpaceManager(BlockSpaceManager):
with num_lookahead_slots. with num_lookahead_slots.
Args: Args:
seq_group (SequenceGroup): The sequence group to swap in. seq_group (SequenceGroup): The sequence group to swap out.
num_lookahead_slots (int): Number of lookahead slots used in num_lookahead_slots (int): Number of lookahead slots used in
speculative decoding, default to 0. speculative decoding, default to 0.
@@ -409,7 +409,7 @@ class SelfAttnBlockSpaceManager(BlockSpaceManager):
swapping out the given sequence_group with num_lookahead_slots. swapping out the given sequence_group with num_lookahead_slots.
Args: Args:
sequence_group (SequenceGroup): The sequence group to swap in. sequence_group (SequenceGroup): The sequence group to swap out.
Returns: Returns:
List[Tuple[int, int]]: The mapping of swapping block from List[Tuple[int, int]]: The mapping of swapping block from
@@ -459,7 +459,7 @@ class SelfAttnBlockSpaceManager(BlockSpaceManager):
on to the 'device'. on to the 'device'.
Args: Args:
sequence_group (SequenceGroup): The sequence group to swap in. sequence_group (SequenceGroup): The sequence group to swap in/out.
device (Device): device to swap the 'seq_group' on. device (Device): device to swap the 'seq_group' on.
status (SequenceStatus): The status of sequence which is needed status (SequenceStatus): The status of sequence which is needed
for action. RUNNING for swap out and SWAPPED for swap in for action. RUNNING for swap out and SWAPPED for swap in