tests/evals/gsm8k/configs/moe-refactor/Mixtral-8x7B-BF16-fi-cutlass.yaml

model_name: "mistralai/Mixtral-8x7B-v0.1"
accuracy_threshold: 0.58
num_questions: 1319
num_fewshot: 5
server_args: "--enforce-eager --max-model-len 8192 --tensor-parallel-size 2 --enable-expert-parallel"
env:
  VLLM_USE_FLASHINFER_MOE_FP16: "1"
  VLLM_FLASHINFER_MOE_BACKEND: "throughput"
[MoE Refactor][17/N] Apply Refactor to Bf16 (#31827) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> 2026-01-15 12:53:40 -08:00			`model_name: "mistralai/Mixtral-8x7B-v0.1"`
			`accuracy_threshold: 0.58`
			`num_questions: 1319`
			`num_fewshot: 5`
			`server_args: "--enforce-eager --max-model-len 8192 --tensor-parallel-size 2 --enable-expert-parallel"`
			`env:`
			`VLLM_USE_FLASHINFER_MOE_FP16: "1"`
[NVIDIA][test] Tests for flashinfer TRTLLM BF16 MoE (#33715) Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com> Co-authored-by: Pavani Majety <pmajety@nvidia.com> 2026-02-11 13:38:11 +01:00			`VLLM_FLASHINFER_MOE_BACKEND: "throughput"`