Artificial intelligence news.
Straight from the original announcements.

d-Matrix says Corsair cuts Qwen3 235B latency to 0.635 seconds

November 16, 2026, 12:43By the AI News Bites Editorial TeamSource: d-Matrix
d-Matrix says Corsair cuts Qwen3 235B latency to 0.635 seconds
Image: d-Matrix

d-Matrix says its Corsair accelerator can speed up speculative decoding by moving the draft model from GPUs to dedicated hardware. In a benchmark using Qwen3 235B, the company reports 476.4 tokens per second per user and 0.635 seconds of end-to-end latency.

Key points
  • Corsair runs the small draft model while GPUs verify results.
  • d-Matrix reports 0.635 seconds of end-to-end latency for Qwen3 235B.
  • The benchmark reached 476.4 tokens per second per user.
  • The Corsair setup used 2x Corsair cards alongside 2x H200 GPUs.
  • d-Matrix Demo Cloud is generally available to select customers.

Corsair separates drafting from GPU verification

Speculative decoding uses a smaller model to propose tokens before a larger model checks them. In d-Matrix’s heterogeneous pipeline, GPUs handle the prefill phase and the verification pool, while Corsair runs the speculative decode pool.

The company’s setup uses Qwen3-1.7B on two Corsair cards to produce draft results. A GPU then selects the best result and re-seeds the draft model. d-Matrix says the Corsair speculator runs an independent inference loop with its own KV Cache.

The approach targets a limitation in small-model inference, where model weights and KV-cache data must move through memory quickly. According to d-Matrix, Corsair’s on-chip SRAM provides 150 TB/s of memory bandwidth, compared with 3.35 TB/s for standard HBM3e.

Benchmark shows lower latency than GPU-only alternatives

d-Matrix compared four serving scenarios for Qwen3 235B: Corsair-assisted speculative decoding, EAGLE3, GPU/GPU standalone speculative decoding and a no-speculation baseline. Each scenario used 2x H200 GPUs for the target model, while the Corsair configuration added 2x Corsair cards for the draft model.

At its best operating point, Corsair-assisted speculative decoding reached 476.4 tokens per second per user, with a 2.10-millisecond TPOT and 0.635 seconds of end-to-end latency. The company reports a 5.48x TPOT speedup versus the no-speculation baseline.

  • EAGLE3 reached 210.5 tokens per second per user, with 1.311 seconds of end-to-end latency.
  • GPU/GPU standalone speculative decoding reached 181.8 tokens per second per user, with 1.502 seconds of end-to-end latency.
  • The no-speculation baseline reached 87.0 tokens per second per user, with 3.033 seconds of end-to-end latency.

The benchmark produced 256 output tokens per request, with average time to first token of 100 milliseconds across all scenarios. Corsair’s measured draft step time was 0.3 milliseconds.

Longer lookahead helped Corsair’s draft model

The tests swept lookahead lengths from 5 tokens to 64 tokens. Corsair’s mean accepted length increased from 4.5 tokens at a lookahead of 5 to 16.0 tokens at lookaheads of 56 and 64. Its best result came at a lookahead of 32, where the system reached 476.4 tokens per second per user.

d-Matrix says Corsair stayed above 460 tokens per second per user from lookahead 24 through lookahead 56. Beyond lookahead 32, the extra draft cost outweighed the small gains in mean accepted length. By comparison, EAGLE3’s mean accepted length remained between 3.8 and 4.55 across the tested range.

Deployment uses existing GPU infrastructure

The reference architecture pairs a GPU node with 2-4x H200s and a Corsair node with 8 Corsair cards. The GPU node runs SGLang with the d-Matrix plugin and exposes an OpenAI-compatible HTTP endpoint, while the Corsair node runs the draft model on native d-Matrix hardware acceleration.

The nodes communicate over TCP, using gRPC for control and plain TCP for data. d-Matrix says the minimum deployment is one GPU server plus one Corsair server, with traffic on a single onboard 1G/10G Ethernet link. RDMA NICs are not needed today, according to the company.

Clients can use SGLang’s OpenAI-designed Chat Completions and Responses APIs by pointing the OpenAI SDK’s base_url at the GPU host. d-Matrix also says its Demo Cloud is generally available to select customers. The results position Corsair as an add-on for data centers already running GPUs, although the reported performance figures come from the company’s stated benchmark configuration.

Related news

A CatBoost model predicts fine-tuning time far better than FLOPs
ResearchNovember 18, 2026, 19:53

A CatBoost model predicts fine-tuning time far better than FLOPs

A CatBoost model predicted the end-to-end duration of fine-tuning jobs far more accurately than conventional FLOP calculations in a Hugging Face study of 272 production…

d-Matrix Named a Top 5 Vendor to Watch by HPCwire
Chips and infrastructureNovember 17, 2026, 01:01

d-Matrix Named a Top 5 Vendor to Watch by HPCwire

d-Matrix has been named one of the Top 5 Vendors to Watch in the 23rd annual HPCwire Readers' and Editors' Choice Awards. The recognition highlights the company's…

d-Matrix Opens Demo Cloud for Corsair Inference Evaluations
Products and toolsNovember 16, 2026, 12:32

d-Matrix Opens Demo Cloud for Corsair Inference Evaluations

d-Matrix has made its hosted Demo Cloud available to selected customers, partners and evaluators, offering a way to test Corsair-powered inference on production-grade…