d-Matrix pairs Corsair with Nvidia H100 for low-latency attention-FFN split

How the attention-FFN disaggregation works

The technique is called attention-FFN disaggregation (AFD). In d-Matrix's design, an Nvidia GPU manages attention, embeddings and expert routing, while Corsair runs the expert computation using its SRAM-based memory architecture. The company says it had already shown speculative decoding working with Corsair and GPUs.

The test model is GPT-OSS-20B, which d-Matrix describes as a lightweight mixture-of-experts (MoE) model. According to the company, the experts make up roughly 19.12B of its 20.9B parameters. At 4-bit precision they fit on an eight-card Corsair server with 16GB of total SRAM. The model activates 4 of 32 experts per token, roughly 3.6B active parameters.

One 80GB H100 handles the attention layer, the KV cache, embeddings and top-4 expert routing. Instead of fetching expert weights from HBM at every step, the GPU sends the hidden state to Corsair and receives the feed-forward results back, which d-Matrix says requires two transfers rather than a large series of memory trips.

JetStream and the LM head

The H100 and the eight Corsair cards are linked through JetStream, which d-Matrix describes as its latency-optimized transparent NIC. It provides Ethernet-based communication with a network bandwidth of 400 Gb/s per cluster. It also supports device-initiated communication: the GPU kernel writes data directly into Corsair memory, and Corsair writes results straight back into GPU memory, with no host CPU involved.

GPT-OSS-20B has 24 layers, so the pipeline makes 24 round trips between Corsair and the H100, plus one more for the LM head. The company says Corsair processes the feed-forward network while the GPU pre-fetches weights for the next layer, so the pipeline only perceives a minimal exposed share of the round-trip latency.

Each Corsair card holds one shard of the LM head, streamed in from DRAM while the remaining layers complete. The GPU then picks the best of the eight proposed results. According to d-Matrix, this cut the LM head's time by 10x and largely hides the off-chip memory traffic.

Reported performance figures

d-Matrix compares the setup with a single H100 running GPT-OSS-20B at a 1K context length, which delivers a TPOT (time per output token) of 2.59ms. With Corsair in the AFD pipeline, the company reports 1.28ms, a roughly 50% reduction. Measurements are dated 10/10/2026.

At a batch size of 64, two H100 GPUs share the same eight Corsair cards, each GPU running 32 user requests. The company says a full 64-user decode step takes 2.46ms at 129 context length. It reports total throughput of 20,413 tokens per second across all users at a 1K/1K context length. The introduction of the post gives the figure as 20,415 tokens per second, so the source is not consistent on this number. A side-by-side comparison with GPU-only setups at batch size 64 is promised in a later blog.

The company attributes the gains to memory bandwidth. It says Corsair's SRAM offers 150 TB/s, which it puts at 45x the bandwidth of standard HBM3 on the H100 (about 3.35 TB/s) and roughly 9x the 18TB/s of HBM4.

What it means

d-Matrix argues the split is natural because the FFN is a fixed-memory object, so Corsair's workload depends on batch size rather than context length, while the GPU takes on the components that grow with context. The company says Corsair's air-cooled, PCIe-based design fits standard data center configurations.

It also says Corsair can scale to models in the hundreds of billions of parameters, targeting models as large as GLM 5.x and Kimi K3, and that AFD and speculative decoding can work together. All figures come from d-Matrix's own tests and have not been independently verified. The company's pitch is that adding GPUs to hold latency leaves much of their compute unused, and that a disaggregated approach with memory-optimized accelerators is an alternative.