Astera Labs Says Modularity Is Key to Building Inference Capacity Fast
Astera Labs says modularity at every level, from chip to grid, is how the industry can build inference capacity fast enough to meet demand. In a blog post tied to the 2026 OCP Global Summit, the company also points to its Orion Rack-Scale Reference Architecture as a response.
- Astera Labs argues modularity from chip to grid is how inference capacity gets built fast enough.
- The post cites Lisa Su's 14x rise in agentic traffic and a 7x jump in Google's token consumption through May.
- The Open Adapter Card is described as the OAM successor, supporting up to 5 kW per module with direct liquid cooling.
- The Orion Rack-Scale Reference Architecture is compute-agnostic and scales from 8 to 160 GPUs, according to Astera Labs.
- Crusoe urged the industry to move data center construction "from a project to a product".
Inference demand outpaces construction
The post, written by Chris Petersen, Vice President and Technologist in Astera Labs' Office of the CTO, opens with a simple claim: inference demand is growing faster than the industry can build for it. According to the post, the theme ran through the 2026 OCP Global Summit.
It cites AMD CEO Dr. Lisa Su, who called 2026 "the year of inference," with agentic traffic up 14x this year. Dr. Dimitrios Ziakas, CTO of Hyve Solutions, pointed to a 7x year-over-year jump in Google's token consumption through May. Petersen concludes that workloads, silicon and data center infrastructure must now evolve at the same pace, and that a data center design fixed for 20 years, or even three, is now a liability.
Astera Labs lists three benefits of designing for modularity: reusability, configuration flexibility and speed to market. Common building blocks work in many places, proven components adapt quickly to new configurations, and each successful deployment builds confidence in the next, so teams spend less time re-validating each build.
Modularity from chip to grid
The post describes how OCP sets standards "from chip to grid", and argues that this vision is impractical without modularity because no single vendor owns every layer between a chiplet and a data center campus.
- Chip: OCP's Open Chiplet Economy aims to create repeatable standards and interfaces at the silicon level. Standardizing the physical envelope and interfaces of an I/O chiplet would enable vendor optionality, since many accelerators have similar interconnect requirements.
- Board and system: Astera Labs' Christopher Blackburn spoke about the Open Adapter Card (OAC), the successor to the now-7-year-old Open Accelerator Module (OAM). The OAC is a vertical card with a backplane connector, about 1.75x the PCB area of a full-height, full-length PCIe card, and up to 5 kW per module with direct liquid cooling.
- Rack and pod: AMD's Helios and Helion racks are built on the standard now called Open Rack Wide. The Open Data Center for AI initiative is standardizing pod dimensions, inter-rack cabling specs and rack footprint spacing. The post also mentions the Mt. Diablo power rack sidecar, co-developed by Microsoft, Google and Meta.
- Building: Facilities must be rapidly repeatable structures. The post cites Meta's announced plans for tent-like data center structures that it proposes to stand up in only 3-5 months each.
A common software layer ties these parts together, the post says, giving operators a single view to orchestrate, manage and configure them.
From project to product, and the Orion architecture
Alex Akesson and Bill Stube of Crusoe called on the industry to transform data center construction "from a project to a product" in their talk on factory-built units. Petersen says this is where the industry has the furthest to go, and that every modular rack needs a standardized playbook for scale-up physical implementation, as well as for pod- and building-level deployment.
He also notes that IDC's Chris Drake argued the emphasis must move from open building blocks to integrated reference architectures, with success measured by interoperability among competing suppliers. Petersen adds that the kind needed today are rack-scale reference implementations.
Astera Labs positions its Orion Rack-Scale Reference Architecture as that kind of design. According to the company, it is compute-agnostic and built to accommodate varied XPU and CPU options. Reference designs cover compute trays, fabric trays, memory trays, compute boards and link boards, enabling a system that scales from 8 to 160 GPUs. The Orion Compute Board incorporates a Scorpio X-Series 320L Smart Fabric Switch, which the company says delivers up to 89% more tokens per s; the source text is cut off at that point.
What it means for operators
The argument is aimed at operators who cannot start from scratch for every deployment, especially inference cloud providers outside the hyperscalers. For them, Astera Labs says proven, flexible components and standardized implementation playbooks can shorten time to market and keep total cost of ownership under control. Whether Orion delivers on that promise will depend on how widely the reference architecture is adopted, which the source does not address.
