Add fused top-K softmax kernel for MoE (#2769)

This commit is contained in:
Woosuk Kwon
2024-02-05 17:38:02 -08:00
committed by GitHub
parent 2ccee3def6
commit f0d4e14557
9 changed files with 591 additions and 50 deletions

9
csrc/moe/moe_ops.h Normal file
View File

@@ -0,0 +1,9 @@
#pragma once
#include <torch/extension.h>
void topk_softmax(
torch::Tensor& topk_weights,
torch::Tensor& topk_indices,
torch::Tensor& token_expert_indices,
torch::Tensor& gating_output);