Baseten says kernel tuning pushes DeepSeek V4.1 Flash to 991 tokens/s
Baseten says it optimized more than 80 kernels and bottlenecks in DeepSeek V4.1 Flash 1017, creating a Fast variant that reached 991 output tokens per second on Artificial Analysis. The company used kernel fusion, Programmatic Dependent Launch and host loop minimization to reduce inference overhead.
- Baseten optimized 80+ kernels and bottlenecks in DeepSeek V4.1 Flash 1017.
- The Fast variant reached 991 output tokens per second on Artificial Analysis.
- Kernel fusion kept intermediate values on-chip and cut a router kernel's decode time by 2.7ms.
- Baseten says its mHC kernel work increased speed by 30% at batch size 1.
- The company plans open-source work on the three optimization techniques.
A faster version of a cost-focused model
Baseten’s work targets DeepSeek V4.1 Flash 1017, which the company describes as a cost-effective, high-quality open model. At native precision, the model is a 552B-parameter mixture-of-experts system, with 8B parameters active during prefill and 16B active during decode.
According to Baseten, DeepSeek V4.1 Flash 1017 scores 32 on the Artificial Analysis V5.0 Index, the same score as GLM-5.3. The company says it delivers that score at approximately one-third the cost and four times the speed of the larger model. Baseten also says the model is on the price and intelligence Pareto frontier.
The company positions flash models as suitable for execution and implementation subagents handling token-heavy, multi-turn tasks. These tasks can be delegated by main agents that run on more powerful and costly models, according to Baseten.
How kernel fusion cuts inference work
Baseten’s first optimization is kernel fusion. In CUDA, a kernel is a user-defined function that runs on NVIDIA GPUs. When separate kernels pass intermediate results between them, those results may make an unnecessary trip through video memory, adding data movement and launch overhead.
A fused kernel combines multiple operations so intermediate values remain in on-chip memory. Baseten gives QK-norm as a simple example: instead of separately normalizing Q and K and then applying RoPE, the fused version reads the inputs once, performs the operations, and writes the final tensors.
The company also worked on fused kernels for mixture-of-experts routing and for the mHC, or Manifold-Constrained Hyper-Connection, layer. Baseten says its improved fused router kernel reduced decode time by 2.7ms, while its work on the open-source mHC megakernel in DeepInfer increased speed by 30% at batch size 1.
Reducing gaps between dependent kernels
Kernel fusion reduces the number of launches and the amount of input and output work, but decoding still requires hundreds of kernels to run in sequence. Baseten used Programmatic Dependent Launch, or PDL, to reduce idle time between dependent operations.
Without PDL, a later kernel waits for an earlier kernel to finish and write its results to memory. With PDL, the earlier kernel signals that the next one can launch. The next kernel can then perform independent work before waiting for the data it depends on.
Baseten says PDL on Blackwell and Vera Rubin can increase overlap between kernels. On B200 systems, the dependency is grid-granular. Vera Rubin supports a finer-grained dependency, allowing a consumer thread block to begin dependent work as soon as the required data arrives.
The post says Baseten used an internal harness to optimize 80+ kernels and bottlenecks. The company plans to release open-source work on kernel fusion, PDL and host loop minimization.
