biondizzle
f3b551956d
- Move dead dsv4/ modules to dsv4/_archive/ (52 files)
- model/{dsv4,mtp,layer,layer_schedule}
- layers/{embedding,attention,ffn,norm} (kept linear,mhc,router,moe,shared_expert,grouped_linear - live)
- cache/*, kernels/cache/*, kernels/indexer/{csa_indexer,score_topk,compute_valid_lens}
- kernels/router/{nvfp4_fused_router,dense_router_decode_kernel,dense_router_prefill}
- ops/{topk,topk_select,rope,router}, loader/{hf_checkpoint,layout_convert}
- reference/{attention,compressor,csa_attention,moe_pipeline}
- kernels/compressor/{compress_tail,csa_hca}
- Restore dsv4/ops/{router,custom_ops}.py (needed by live layers)
- Fix dsv4/kernels/{indexer,compressor,attention}/__init__.py (removed broken imports)
- Remove preload_all() from loader.py (dead, referenced nonexistent .cu file)
- Fix loader.py docstring (fused_amax_quantize_nvfp4 → quantize_nvfp4_from_buffer)
- Move broken tests to tests/e2e_archive/
- test_fused_router, production_values_test, e2e/{one_layer,model_construction,csa_hca}
- vLLM has 0 imports of dsv4 (Step 0 confirmed)
27 lines
1.1 KiB
Python
27 lines
1.1 KiB
Python
"""Compute per-query valid compressed entry count from block table.
|
|
|
|
Small integer reduction: for each request, valid_len = block_lens * entries_per_block
|
|
accounting for the partially-filled last block. Used by the indexer score kernel
|
|
to know how many candidate keys to stream.
|
|
"""
|
|
import torch
|
|
|
|
|
|
def compute_valid_lens(
|
|
block_lens: torch.Tensor, # [B] int32 — number of blocks per request
|
|
block_table: torch.Tensor, # [B, max_logical_blocks] int32
|
|
entries_per_block: int,
|
|
) -> torch.Tensor:
|
|
"""Return [B] int32 — total valid compressed entries per request.
|
|
|
|
For now, a simple formula: valid_entries = block_lens * entries_per_block.
|
|
This assumes all entries in all allocated blocks are valid, which is correct
|
|
because blocks are only allocated when flush writes to them, and each block
|
|
is fully populated before the next is allocated (compression ratio is fixed).
|
|
|
|
In a more general design with partially-filled blocks, this would need
|
|
to check the actual write positions. For DSV4's fixed-ratio compression,
|
|
the simple formula is exact.
|
|
"""
|
|
return block_lens * entries_per_block
|