[Misc] skip target model mm emb in draft proposal step when draft is text-only (#33437)
Signed-off-by: kkt-cohere <komal@cohere.com>
This commit is contained in:
@@ -4070,7 +4070,7 @@ class GPUModelRunner(
|
||||
else:
|
||||
target_hidden_states = hidden_states[:total_num_tokens]
|
||||
|
||||
if self.supports_mm_inputs:
|
||||
if self.supports_mm_inputs and self.drafter.supports_mm_inputs:
|
||||
mm_embed_inputs = self._gather_mm_embeddings(
|
||||
scheduler_output,
|
||||
shift_computed_tokens=1,
|
||||
|
||||
Reference in New Issue
Block a user