

DeepSeek's DeepInfer is now open source under the Apache 2.0 license, exposing the custom FlashMLA, DeepGEMM, and DeepEP kernels behind DeepSeek-V4.1-Flash serving.
Multi-vendor kernels, offline scope
DeepInfer targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs with asynchronous batching, grammar-constrained outputs, and DSpark speculative decoding. Out of the box, it is an offline batch engine—developers deploying online web APIs must add their own HTTP server, KV-cache disaggregation, and load balancing.
Original reporting; primary project releases reviewed November 16, 2026. No independent hands-on benchmark claimed.



