vLLM Supports Reflection's 501B-Parameter Beam Model From Day 0

Full feature support and a ready Docker image

According to vLLM, reasoning, tool calling and structured outputs work out of the box with Beam. The model can be run with tensor, data or expert parallelism. The team publishes a Docker image, vllm/vllm-openai:reflection-beam, and a default command that serves reflection/Beam-1.0-501B-A23B-NVFP4 with TP4 on a single GB300 node, using DFlash2 speculative decoding and an FP8 KV cache.

The command enables Beam-specific reasoning and tool-call parsers, sets the maximum number of sequences to 128 and selects the FlashInfer attention backend.

How Beam is built

Beam is a 52-layer MoE decoder with hidden size 5120. Its expert routing follows DeepSeek-V3: 256 routed experts, with the best 4 of 8 expert groups chosen first and then the top 8 experts. Unlike DeepSeek-V3, it has no shared expert, and layers 0 to 2 use a dense SwiGLU MLP instead. The router runs entirely in FP32.

The main differences lie in attention and the residual stream:

• Hybrid attention: 43 layers use sliding-window attention over the last 512 tokens with RoPE. The other 9 attend over the full context with no positional encoding. On those layers, SSMax scales queries so attention sharpens as context grows.
• Gated, normalized heads: 48 query heads and 8 KV heads, with per-head Q/K normalization and a sigmoid output gate that can turn a head down or off token by token.
• Sandwich norm and depth-scaled residuals: every sublayer has an RMSNorm before and after it, and residual adds are scaled so the stream keeps unit variance at every depth.

Because only the 9 full-attention layers keep KV for the whole sequence, the cache stays small. vLLM puts it at 36 KiB per token in BF16 and 18 KiB in FP8, or about 18 GiB of FP8 KV for a full 1M-token context, before sharding.

Kernels tuned by coding agents

vLLM's first Beam implementation was an almost pure PyTorch version. The team then let coding agents profile the model, write kernels and benchmark each change. On TP8, seven changes cut batch-1 decode from 8.95 ms to 3.30 ms per step, and an 8 × 2K prefill from 328 ms to 176 ms. vLLM describes this as 2.71× faster decode and 1.87× faster prefill than its first implementation.

The changes include a single Triton kernel for the post-norm, scaled residual add and next pre-norm at each of the 104 sublayer boundaries, which accounts for a 40% decode reduction. Others fuse Q/K norm, RoPE or SSMax and the output gate, run the tensor-parallel all-reduces inside the norm kernels using FlashInfer's multicast all-reduce, add a specialized FP32 router kernel, pack the Q, K, V and gate projections into one GEMM, and write K and V into the cache from the same kernel. After this work, one MoE layer runs as 13 kernels in batch-1 decode on TP8.

Attention DP with MegaMoE for high concurrency

For high-concurrency workloads, vLLM combines Attention data parallelism with FlashInfer's CuTe DSL NVFP4 MegaMoE kernel, which can fuse communication with expert computation. On 4×GB300, with 32K input tokens and a 1-token output to isolate prefill, this setup reached a peak of 168.8K input tokens per second, against 98.7K for the TP configuration. vLLM puts the gain at 1.71× higher peak prefill throughput than TP4.

It is enabled with --moe-backend flashinfer_moe_ep_cutedsl, --data-parallel-size 4 and --enable-expert-parallel.

DFlash2 and adaptive verification

Beam ships with a DFlash2 draft head. Instead of drafting one token at a time, it predicts a whole block of tokens in one forward pass from the target model's hidden states. The drafter has 5 sliding-window layers that read hidden states from 5 target layers, and it shares the target's embedding and LM head. FP8 KV cache works for the drafter with no change in acceptance, according to vLLM.

With adaptive verification switched on, vLLM estimates how likely each draft token is to be accepted and verifies only those worth the cost. Because DFlash2 has no confidence head, the estimate comes from the drafter's own logits.

vLLM tested on TP8 using the low-entropy category of the SPEED-Bench throughput_1k split, mostly code completion and text sorting with greedy decoding. Results:

• Concurrency 1: 708 tok/s with fixed drafts, 707 tok/s adaptive (−0.1%).
• Concurrency 8: 3,232 versus 3,221 tok/s (−0.3%).
• Concurrency 32: 7,744 versus 8,275 tok/s (+6.9%).
• Concurrency 128: 15,012 versus 16,632 tok/s (+10.8%), with p99 TPOT cut by 9%.

vLLM cautions that low-entropy text is easier to predict, so acceptance will be lower on open-ended chat.

What it means for users

Teams that want to run Beam can do so on vLLM immediately, with parsers for reasoning and tool calls already in place. The reported figures come from vLLM's own tests on NVIDIA Blackwell hardware such as GB200 and GB300. Adaptive verification gives its clearest benefit under heavy load and is neutral at low concurrency, so it matters most for busy serving deployments.