

Tencent's LightningMoBA combines changes to sparse-attention selection with changes to GPU scheduling. The November 19 report studies that combined design rather than treating a smaller attention workload as an automatic proportional speedup.
Mixture of Block Attention selects groups of key-value tokens for a query. LightningMoBA replaces a fixed per-query retention budget in its main method with threshold-based selection, then groups work around key-value blocks to reuse loaded data more effectively.
What the reported comparison measures
The authors compare against FlashMoBA at matched average retention budgets. Their main RULER evaluation covers 6,500 examples, six context lengths from 4K to 128K, tensor parallelism of eight and batch size one. The reported end-to-end improvement range is 1.20–1.60 times across five checkpoints.
The guide clarifies that timing covers the complete generation loop, not isolated request latency. Larger routing-only or attention-path gains reported separately should not be substituted for that end-to-end figure. One checkpoint is internal, another reason to read reproducibility details alongside the summary.
Compatibility is part of the result
The default KV-centric pipeline requires SM90 hardware and CUDA 12.3 or later, with compatible PyTorch and build tools. The documented dynamic paths use BF16, head dimension 128 and 128-token blocks, and are causal inference-only paths.
A fixed Top-K baseline provides backward support through a separate entry point. That does not mean the headline dynamic pipeline supports the same training use. Unsupported inputs on the main path raise an error rather than silently becoming the baseline.
Our analysis: selected blocks are only part of the cost
Routing, sparse-index construction and irregular data movement can consume work that a high-level sparsity count does not show. The contribution is the attempt to design these pieces together. Timing only the final attention arithmetic can miss much of that behavior.
Matching hardware and input shape is therefore as important as matching a model size. A multi-GPU research result cannot establish an improvement in a desktop launcher or a different serving stack. Runtime integration and a representative workload remain separate evaluations.
Our profile records the BSD-3-Clause license read from source, despite GitHub's less-specific automatic label. Weights, captures and dependencies have separate terms; benchmark captures are not bundled. We did not compile the library, download checkpoints or run GPU tests.
EXPLORE THE SOURCE
Projects in this story
GitHub repositories
- Tencent/LightningMoBA ↗View project details
Original primary-source reporting with editorial analysis. No installation, model/data download, inference, upstream suite, paid service call or independent security/performance test.

