vLLM Adds Day 0 Support for NVIDIA's Nemotron 3.5 Super VL Multimodal Model

A multimodal model for always-on agents

According to the vLLM post, Nemotron 3.5 Super VL combines reasoning, instruction following and tool use with understanding of text, images, documents and video in a single model. The aim is to reduce the need to deploy separate text and vision models and to help keep task-relevant context across the steps of a workflow.

The model uses a mixture-of-experts architecture with 120 billion total parameters and 12 billion active parameters. It accepts text, images, documents and video as input and produces text as output. Its listed optimizations are the C-RADIOv4 vision encoder, a hybrid Mamba-Transformer architecture, multi-token prediction and NVFP4 quantization.

Within NVIDIA's lineup, the model is designed as the default high-accuracy agentic execution model with native document, image and video understanding. It complements NVIDIA Nemotron 3.5 Lightning, which targets high-volume text execution, and NVIDIA Nemotron 3 Ultra, which handles the hardest text-only reasoning.

Running it on vLLM

vLLM supplies the serving layer, including continuous batching, prefix caching, speculative decoding and an OpenAI-compatible API. The BF16 checkpoint serves as a straightforward baseline, while NVFP4 is available at launch for environments that can use lower-precision inference.

The post gives setup steps based on the vllm/vllm-openai:v0.31.0 Docker image. An example NVFP4 launch configuration for a single GB200 setup uses the nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 checkpoint with a maximum model length of 114688 and the nemotron_v3 reasoning parser. vLLM says this configuration should be validated on the target hardware before use.

The model includes built-in multi-token prediction heads. During decoding, these heads propose future tokens and the target model verifies them, which reduces the number of sequential generation steps for longer responses. The post shows MTP with five speculative tokens and also with three. Applications can then send prompts, including video URLs, through a standard OpenAI-compatible client and read both the reasoning trace and the final answer.

The listed deployment targets are 1x GB10, B200, B300, RTX PRO 6000, H200 and 2x H100. NVIDIA NIM is named for optimized production serving across cloud and data center environments.

Customization and use cases

Organizations can post-train the model with NVIDIA NeMo using their own data, tools, policies and workflows. According to the post, this lets a general multimodal model become a specialized execution engine for domain-specific agent tasks while preserving control over model behavior and data handling.

The example use cases are:

• Customer service: resolving multi-step requests by interpreting customer documents and images, applying policy and using service tools.
• Document intelligence: interpreting complex documents, including OCR, tables, charts and page layouts, to extract information and generate summaries or reports.
• Enterprise productivity: completing multi-step tasks by interpreting visual information, reasoning across it and using tools.
• Video understanding: extracting key information to support search, content review, advertising, retail and incident analysis.
• Web research: searching across websites, connecting information from multiple sources and verifying results.

Performance claims and what they mean

The post makes several performance claims, which come from the announcement itself. It says that on a single GPU the model delivers up to 5x faster throughput for agentic tasks compared with other open models in its size category, and reaches over 600 output tokens per second at 100,000 input tokens. The throughput figure is attributed to Artificial Analysis Intelligence Index agentic tasks on a Baseten endpoint. A second chart, based on SWE-bench Pro accuracy and the hours needed to complete 10,000 tasks on four B200 GPUs, is described as showing 4x faster task completion than other leading open models of similar size.

The post also includes a DGX Spark chart for the related Nemotron 3.5 Super model, which it stresses is a different model and is not measured in that chart. That model scores 80 on the local.ai intelligence index and takes 6.5 minutes per task on DGX Spark.

For developers, the practical point is that the model can be served through vLLM from day one, with both BF16 and NVFP4 weights available, so teams already using vLLM can test multimodal agent workloads without waiting for later integration. A getting-started cookbook is available from vLLM.