[Frontend][Model] Add 'float16' to possible mamba cache dtype values, override mamba SSM cache dtype value for NemotronH (#29978)

Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
2025-12-05 10:34:33 +02:00
parent 60a66ea2dc
commit 6038b1b04b
3 changed files with 23 additions and 1 deletions
--- a/vllm/utils/torch_utils.py
+++ b/vllm/utils/torch_utils.py
@@ -28,6 +28,7 @@ else:
 STR_DTYPE_TO_TORCH_DTYPE = {
    "float32": torch.float32,
    "half": torch.half,
+    "float16": torch.float16,
    "bfloat16": torch.bfloat16,
    "float": torch.float,
    "fp8": torch.uint8,