NVIDIA Unifies GPU-Initiated Networking Under DOCA GPUNetIO Across Its Stack

Taking the CPU off the critical path

According to NVIDIA, GPU applications increasingly need networking and data movement to act as GPU-controlled operations rather than host-driven services. When the CPU sits in the middle of every network transaction, it becomes a bottleneck on the critical path, adding latency and limiting how well distributed applications can respond in real time.

DOCA GPUNetIO is described as a GPU-centric networking SDK layer for real-time packet processing and data movement. It combines technologies such as GPUDirect RDMA, GPUDirect Async Kernel-Initiated (GDA-KI) and GDRCopy, so that CUDA kernels can directly drive Ethernet, RDMA, Verbs and DMA operations while the CPU stays out of the critical path.

The framework offers two parts: CPU functions on the control path that export transport objects created with DOCA Ethernet, DOCA Verbs, DOCA DMA and DOCA CommChannel into GPU memory, and GPU CUDA functions on the data path that let kernels manipulate those exported objects.

One shared implementation instead of many

NVIDIA says that before GPUNetIO became the common foundation, each communication library had built its own GDA-KI-style, GPU-initiated RDMA implementation, with its own assumptions, code and maintenance burden. None of them shared anything with the others.

The framework is now integrated across NVSHMEM, NCCL, Aerial 5G SDK, UCX/NIXL, NVQLink with the Holoscan Sensor Bridge operator, the Holoscan Advanced Network Operator, DeepEP/HybridEP and others. NVIDIA argues that converging on one GDA-KI implementation creates a shared place for features, optimizations and bug fixes, reduces duplicated engineering across SDKs, and lets innovations from one framework reach others faster.

• NCCL GIN has integrated the open-source GPUNetIO Verbs path as a backend since version 2.27, allowing device-side collective algorithms to drive RDMA operations directly from the GPU.
• NVSHMEM 3.7 introduced a GPUNetIO-based transport that, according to NVIDIA, reduces implementation complexity while preserving IBGDA performance. Benchmarks cited by the company show GDA-KI enables better CTA and QP scaling for small message sizes.
• NVQLink uses the GPUNetIO-based GPU RoCE Transceiver operator to reach approximately 2.6 microseconds minimum round-trip latency for quantum-classical workflows on IGX Thor with a Blackwell GPU and ConnectX-7.

Open source and SDK versions

NVIDIA ships GPUNetIO in two forms. The full DOCA SDK version is the superset, documented in the DOCA Programming Guide, covering Verbs, Ethernet, DMA and Comm Channel integration. The open-source project is a lighter-weight, RDMA-Verbs-focused implementation aimed at frameworks that want to stay fully open in how they integrate networking transports.

The company says the split is not meant to create two diverging stacks. The open-source implementation can detect the DOCA SDK at runtime and, if it is present, call selected closed-source SDK functions through dlopen. This requires a documented environment variable to be set with the correct SDK path. Without the SDK, the open-source version runs standalone. For the Verbs path, both variants expose a similar CUDA device API, which keeps the GPU programming model largely aligned.

Programming model

Applications first run a host-side CPU phase for the control path, then move the data path to CUDA kernels on the GPU. The control path initializes the GPU and network devices and allocates memory, then creates network transport objects. For example, DOCA Verbs or DOCA Ethernet creates a network queue object via mlx5dv on the CPU. A GPUNetIO CPU function then exports the relevant elements of that queue into a descriptor in GPU memory and returns a GPU address for it. The application launches a CUDA kernel and passes that descriptor as an input parameter.

To simplify DOCA Verbs setup, NVIDIA introduced high-level functions such as doca_gpu_verbs_create_qp_hl, which condense the steps for creating and connecting RDMA QPs. They are present in both the SDK and open-source versions. In the data path, CUDA threads post Work Queue Entries (WQEs), such as RDMA Write or RDMA Read.

What it means for developers

For teams building on NCCL, NVSHMEM or UCX/NIXL, the change means GPU-initiated networking improvements are meant to land in one shared codebase rather than in several parallel ones. NVIDIA points to less duplicated code, less fragmentation in behavior and a stronger path for long-term evolution. Developers can start from the DOCA SDK GPUNetIO programming guide, the open-source GPUNetIO repository, and the NCCL tests GIN device API for reference code.