vLLM Adds Day-0 Support for MiniMax-M3.1-Flash on NVIDIA and AMD GPUs

A model built for efficient inference

MiniMax-M3.1-Flash is the latest model in the MiniMax-M3 family, with improvements across reasoning, coding, agentic and visual tasks, according to vLLM. The M3 family replaces full attention with a variant of block-sparse attention, called MiniMax Sparse Attention (MSA).

In MSA, the KV cache is split into blocks of 128 tokens. A lightweight indexer scores every key in the context and pools those scores into one score per block. Main attention then runs exactly, but only over the top 16 blocks, and the most recent block is always kept. That caps each query at 2,048 keys, however long the context is. vLLM says that at long-context decode, per-step HBM traffic drops to about 12.5% of what dense attention reads.

MiniMax-M3.1 was natively quantization-aware trained for an NVFP4 KV cache, and its routed MoE weights are stored in NVFP4. The post says the NVFP4 experts shrink the checkpoint from 451.5 GB (MiniMax-M3-MXFP8) to 250.1 GB, which is 44.6% smaller. The NVFP4 KV cache needs 34,560 bytes per token instead of 61,440 bytes in FP8, a saving of 43.75%. On four NVIDIA B300 GPUs with DEP4 deployment and 256K-token sequences, M3.1 fits 76 concurrent sequences where M3 with an FP8 KV cache fits 28, about 2.7× as many.

Kernel and serving optimizations

vLLM lists several optimizations for the new model:

• Blackwell MSA kernels: they use the new CUDA 13.4 QMUL4 instruction to dequantize NVFP4 into FP8 inside the kernel, giving up to 1.7× faster indexer QK decode and 1.3× faster sparse attention decode.
• NVFP4 MegaMoE: with MXFP8 shared-expert fusion and early TMEM release, it brings 20% lower kernel latency in prefill and 7% higher prefill throughput.
• Indexer decode context parallelism: up to 3.3× faster indexer per sparse layer at 512K context on 4× GB300.
• QKV GEMM, norm, RoPE and KV-insert fusion: 1.1, 1.6× faster prefill at 512 or more tokens and about 19% lower P50 TTFT at TP4 on GB300.
• FlashInfer PrimiTS backend for the DSpark drafter: 1.3, 2× faster than FlashAttention 4 and 5% lower total forward time.

The team also says Rubin (SM107) MSA kernels are integrated into vLLM. On AMD MI350-series GPUs, the NVFP4-native MSA kernel, indexer decode context parallelism, dual-batch overlap and the mono kernel deliver a 2, 3× performance boost across the board, according to vLLM.

For agentic serving, vLLM reports up to 418.9 tok/s per user on NVIDIA GB300 at the latency-sensitive end. On NVIDIA Rubin it cites up to 724.9 tok/s per user and 191.5K total tok/s per GPU. On AMD MI355X it reports up to 133.3M total tokens per dollar of TCO on the SemiAnalysis AgentX benchmark.

Getting started and RL support

Prebuilt images are available for both vendors: vllm/vllm-openai:minimax-m31-flash for NVIDIA and vllm/vllm-openai-rocm:minimax-m31-flash for AMD. The model will only reach the nightly PyPI wheel after the code is reviewed and merged upstream, which usually takes about a week. Full recipes are at vLLM Recipes, and the checkpoint is published as MiniMaxAI/MiniMax-M3.1-Flash.

The post gives example TP4 launch commands for both platforms. On NVIDIA, vLLM recommends the CUDA 13.4 toolkit for the best performance, and the setup uses an NVFP4 KV cache, an NVFP4 indexer KV dtype and the DSpark speculative config with 7 speculative tokens. The AMD example enables indexer decode context parallelism, MXFP8 for MoE quantization and 3 speculative tokens.

vLLM also announces day-0 reinforcement learning support, covering end-to-end RL with veRL, FSDP2 and LoRA on the NVFP4 checkpoint.

What it means for users

For teams running long-context or agentic workloads, the figures from vLLM point to lower memory use per sequence and higher throughput on current and upcoming accelerators from both NVIDIA and AMD. All performance numbers come from vLLM itself and depend on the hardware and configurations it describes. Users who want the model in the standard nightly wheel will need to wait for the upstream merge, while the prebuilt images are available now.