biondizzle 9cdf79fd9c wip: fused SwiGLU kernel scaffold + bridge interleave + plan ...

- fused_swiglu_grouped_mm.py: copypaste of torch_scaled_grouped_mm.py with
  class rename and fused_swiglu/swiglu_limit params added
- bridge.py: added interleave_l1_weights, deinterleave_l1_weights,
  warmup_fused_swiglu_compilation
- Pure-PyTorch interleave invariant passes (A@cat vs deinterleave(A@interleave))
- Standalone GEMM interleave test fails due to kernel-internal N-tiling
  layout (expected, skipping per plan)
- FUSED_EPILOGUE_PLAN.md updated with register layout, amax shuffle plan,
  4-step implementation strategy

2026-05-20 03:04:38 +00:00

..

kernel

wip: fused SwiGLU kernel scaffold + bridge interleave + plan

2026-05-20 03:04:38 +00:00

__init__.py

refactor: copy CuTeDSL kernel into repo with local imports

2026-05-16 02:57:54 +00:00

blackwell_attention.py

Add blackwell_attention module and comprehensive test

2026-05-19 15:30:29 +00:00

bridge.py

wip: fused SwiGLU kernel scaffold + bridge interleave + plan

2026-05-20 03:04:38 +00:00

csa_attention.py

Fix device reference in full_attention_reference

2026-05-19 08:01:31 +00:00

custom_ops.py

Replace autograd.Function with torch.library.custom_op for Dynamo compat

2026-05-19 01:54:48 +00:00

inverse_rope.py

Replace DeepGEMM fp8_einsum with CuTeDSL NVFP4 for wo_a (o_proj)

2026-05-19 02:36:30 +00:00

moe_pipeline.py

Add pipeline test with real model weights, add swiglu_limit to reference moe_pipeline

2026-05-17 18:07:44 +00:00

native_swa_decode.py

fix: handle 3D swa_indices and correct kv_bf16 expand dims

2026-05-20 01:36:27 +00:00

nvfp4_linear.py

fix: root-cause JIT memory corruption myth, add eager warmup, remove _needs_token_refill

2026-05-20 02:08:01 +00:00

runner.py

chore: remove unused _expert_id_range after bincount migration

2026-05-20 02:17:44 +00:00

shared_expert_pipeline.py

Fix garbled shared_expert_pipeline.py: imports/class were merged

2026-05-19 07:18:10 +00:00

wo_a_grouped_linear.py

Fix wo_a: scatter each group's data at correct offset in padded buffer

2026-05-19 02:45:57 +00:00