Clean up: remove PLAN-triton-kernels.md (merged into main)
This commit is contained in:
@@ -1,129 +0,0 @@
|
||||
# Plan: Add Triton Kernels Support for vLLM v0.19.0
|
||||
|
||||
**Date:** 2026-04-06
|
||||
**Status:** Ready for execution
|
||||
**Branch:** `feature/triton-kernels` (to be created)
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
vLLM v0.19.0 container builds successfully but fails at runtime with:
|
||||
```
|
||||
No module named 'triton_kernels.matmul_ogs'
|
||||
```
|
||||
|
||||
The error occurs because:
|
||||
- `triton_kernels` is a library of pre-written Triton kernels (separate from the `triton` compiler)
|
||||
- vLLM v0.19.0 requires it for MoE (Mixture of Experts) operations
|
||||
- Our Dockerfile builds vLLM via `pip build --wheel`, which skips the cmake step that normally fetches `triton_kernels`
|
||||
|
||||
---
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
### What vLLM's cmake build does (that we skip):
|
||||
1. Fetches `triton_kernels` from Triton repo (tag v3.6.0) → copies to `vllm/third_party/triton_kernels/`
|
||||
2. Builds `flashmla`
|
||||
3. Builds `vllm_flash_attn` (we already handle this separately)
|
||||
|
||||
### What's in `triton_kernels`:
|
||||
Located at `python/triton_kernels/triton_kernels/` in the Triton repo:
|
||||
- `matmul_ogs.py` — MoE kernels (the missing module)
|
||||
- `topk.py` — Top-k routing
|
||||
- `swiglu.py` — SwiGLU activation
|
||||
- `tensor.py` — Tensor utilities
|
||||
- `distributed.py` — Distributed ops
|
||||
|
||||
### PyPI confusion:
|
||||
- `triton-kernels` on PyPI (v0.1.0) is from "Kernelize AI" — **NOT what vLLM needs**
|
||||
- The real `triton_kernels` is in the Triton repo itself
|
||||
|
||||
---
|
||||
|
||||
## Solution: Option A — Pip install from Git
|
||||
|
||||
Add a build stage that installs `triton_kernels` directly from the Triton repo:
|
||||
|
||||
```dockerfile
|
||||
FROM build-base AS build-triton-kernels
|
||||
# Install triton_kernels from Triton repo (v3.6.0 matches vLLM's cmake default)
|
||||
RUN pip install git+https://github.com/triton-lang/triton.git@v3.6.0#subdirectory=python/triton_kernels
|
||||
# Copy the installed package to wheels for final stage
|
||||
RUN pip show triton_kernels | grep Location | cut -d' ' -f2 | xargs -I {} cp -r {} /wheels/
|
||||
```
|
||||
|
||||
Then in the final `vllm-openai` stage:
|
||||
```dockerfile
|
||||
COPY --from=build-triton-kernels /wheels/triton_kernels /usr/local/lib/python3.12/dist-packages/triton_kernels
|
||||
```
|
||||
|
||||
### Why this works:
|
||||
- vLLM's `import_utils.py` checks `site-packages` for `triton_kernels` first
|
||||
- Installing it there means vLLM will find it
|
||||
- Minimal change, doesn't affect existing working components
|
||||
|
||||
---
|
||||
|
||||
## Execution Steps
|
||||
|
||||
### 1. Create new branch
|
||||
```bash
|
||||
cd /home/openclaw/dev/grace-gpu-containers
|
||||
git checkout -b feature/triton-kernels
|
||||
```
|
||||
|
||||
### 2. Modify Dockerfile
|
||||
Edit `vllm/Dockerfile`:
|
||||
- Add `build-triton-kernels` stage after `build-triton`
|
||||
- Copy `triton_kernels` to final stage
|
||||
- Update header comment with new version info
|
||||
|
||||
### 3. Update CLAWMINE.md
|
||||
Document the new build configuration.
|
||||
|
||||
### 4. Commit and push
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "Add triton_kernels for MoE support (vLLM v0.19.0)"
|
||||
git push origin feature/triton-kernels
|
||||
```
|
||||
|
||||
### 5. Create new Jenkins pipeline
|
||||
Create `gh200-vllm-tfa-build`:
|
||||
- Same as `gh200-vllm-build` but:
|
||||
- Pulls from `feature/triton-kernels` branch
|
||||
- Default `IMAGE_TAG=gh200-vllm-tfa`
|
||||
- Default `VLLM_VERSION=v0.19.0`
|
||||
|
||||
### 6. Trigger build
|
||||
Wait for Mike's OK before triggering.
|
||||
|
||||
---
|
||||
|
||||
## Tag Strategy
|
||||
|
||||
| Image | Tag | Purpose |
|
||||
|-------|-----|---------|
|
||||
| `gh200-vllm` | `v0.19.0` | Working fallback (no triton_kernels) |
|
||||
| `gh200-vllm-tfa` | `v0.19.0-tfa` | New build with triton_kernels |
|
||||
|
||||
If successful, `gh200-vllm-tfa:v0.19.0-tfa` becomes the production image.
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If the build fails or runtime issues occur:
|
||||
1. The existing `gh200-vllm:v0.19.0` image is untouched
|
||||
2. Just revert to using that tag
|
||||
3. No changes to main branch until verified working
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- vLLM cmake config: `cmake/external_projects/triton_kernels.cmake`
|
||||
- vLLM import logic: `vllm/utils/import_utils.py`
|
||||
- Triton repo: https://github.com/triton-lang/triton
|
||||
- Triton v3.6.0 tag: `python/triton_kernels/triton_kernels/matmul_ogs.py`
|
||||
Reference in New Issue
Block a user