Artificial intelligence news.
Straight from the original announcements.

AI inference capacity grew 4.6× in one year, Epoch AI finds

November 18, 2026By the AI News Bites Editorial TeamSource: Epoch AI
AI inference capacity grew 4.6× in one year, Epoch AI finds
Image: Epoch AI

The estimated agentic inference capacity of AI chips sold to date grew 4.6× between Q2 2025 and Q2 2026, according to Epoch AI. That was faster than the 2.6× increase in the chips’ peak 8-bit computing power.

Key points
  • Agentic inference capacity grew 4.6× from Q2 2025 to Q2 2026.
  • Peak 8-bit computing power grew 2.6× over the same period.
  • Blackwell and Blackwell Ultra supplied 74% of added capacity.
  • Inference capacity per unit of peak compute rose about 77%.
  • Epoch AI expects the growth rate to slow as the installed base expands.

Blackwell drove most of the increase

Epoch AI attributes the growth largely to Nvidia’s transition from Hopper to Blackwell. Nvidia’s Blackwell and Blackwell Ultra families accounted for 74% of the added agentic inference capacity during the period, according to the research.

By Q2 2026, those two families represented roughly 73% of total agentic inference capacity, compared with 50% of total peak 8-bit computing power. The report says GB300 systems deliver roughly 8× the inference capacity of H200s per GPU.

The estimate measures cumulative capacity from chips sold to date, rather than capacity added in a single quarter. The total reached 4.58 times its Q2 2025 level in the data, while peak 8-bit compute reached 2.59 times its starting level.

How Epoch AI measured agentic capacity

Epoch AI used version 1 of its Inference Performance Index, or IPI, to compare hardware on a fixed set of workloads with long, cached contexts. The index is expressed in GB300 equivalents and is applied to estimates from Epoch AI’s AI Chip Sales dataset.

The measure accounts for memory, inter-chip communication and lower-precision computation. It includes both input processing and output generation, and uses the same workload basket for every quarter.

To estimate total capacity, Epoch AI multiplied the cumulative number of chips sold in each category by its per-chip IPI score, then summed the results. It calculated peak 8-bit computing power for the same chips as a comparison.

Epoch AI estimates that inference capacity per unit of peak compute rose by about 77% over the year. The organization uses “hardware, model co-design” broadly for gains beyond proportional increases in arithmetic throughput and memory bandwidth, including larger scale-up domains, lower-precision computation and model architectures that use hardware more effectively.

Growth is expected to slow, but the gap may persist

Chips sold in Q2 2026 added 3.3× the inference capacity of chips sold in Q2 2025. Because that addition was smaller than the 4.6× growth in cumulative capacity, Epoch AI expects the pace of total growth to slow as the installed base becomes larger.

The report says sustaining 4.6× annual growth would require yearly additions equal to 3.6× the capacity at the beginning of each year. It also expects smaller improvements in later chip generations. Modeled inference capacity per GPU rises 8× from H200 to GB300, but 2.2× from GB300 to Vera Rubin VR200.

Epoch AI says continued hardware, model co-design should nevertheless keep inference capacity growing faster than peak computing power. Its uncertainty analysis gives 90% intervals of 4.2, 5.1× for inference growth and 2.5, 2.7× for peak compute growth, covering shipment and hardware-mix uncertainty.

Related news

A CatBoost model predicts fine-tuning time far better than FLOPs
ResearchNovember 18, 2026, 19:53

A CatBoost model predicts fine-tuning time far better than FLOPs

A CatBoost model predicted the end-to-end duration of fine-tuning jobs far more accurately than conventional FLOP calculations in a Hugging Face study of 272 production…

NVIDIA Vera Rubin NVL72 reaches up to 2x GB300 performance in MLPerf
Chips and infrastructureNovember 16, 2026, 17:00

NVIDIA Vera Rubin NVL72 reaches up to 2x GB300 performance in MLPerf

NVIDIA’s Vera Rubin NVL72 delivered the fastest training time on six of seven MLPerf Training v6.1 benchmarks, with up to 2x more performance per GPU than an optimized…

NVIDIA cuBLAS adds FP64 algorithms for faster scientific simulations
Chips and infrastructureNovember 13, 2026, 23:13

NVIDIA cuBLAS adds FP64 algorithms for faster scientific simulations

NVIDIA has added two algorithms to cuBLAS that accelerate double-precision matrix multiplication on integer Tensor Cores while preserving full FP64 accuracy. The company…