Meta Andromeda Now Scores Over One Billion Ads Per Second in Retrieval

What Andromeda does and what changed

Meta Andromeda is the machine learning system behind personalized ads retrieval, the first stage of Meta's ads recommendation pipeline. Retrieval narrows a vast pool of eligible ads to a small set of strong candidates in milliseconds. A later, more computationally expensive ranking stage then scores those candidates in detail.

According to Meta, the deep modeling quality of the ranking stage has historically been too costly to apply at retrieval's scale. The new Andromeda aims to close that gap. Meta says the production system now scores every eligible ad with near ranking-level model performance and surfaces the top candidates at over one billion ads per second. The company calls this a new throughput benchmark for retrieval-stage models.

For advertisers, Meta says this means better performance automatically, because the retrieval system can match ads across a larger candidate pool within the same latency budget. Meta also states that since 2025, Andromeda has delivered a 2.4% increase in conversions on Facebook.

Why the 2024 design had to evolve

The 2024 Andromeda launch ran on the NVIDIA GH200 Grace Hopper Superchip and grew model capacity by four orders of magnitude, according to Meta. Since then, generative AI creative tools have helped advertisers create hundreds of millions of new ads, which has expanded Meta's candidate pool to an unprecedented scale.

Meta describes three pressure points:

• Model quality: small gains in retrieval performance can produce large gains in advertiser value, but the 2024 architecture could not keep improving at a sustainable compute cost.
• System throughput: the 2024 system evaluates tens of millions of ads per second per GPU card, while the new demand requires multi-billion ads per second per card.
• Memory bandwidth: at that rate, intermediate tensors between GPU kernels account for terabytes per second of memory traffic, and shipping ads between local CPU hosts and remote accelerators saturates the network link.

Meta says addressing these limits motivated a joint collaboration with NVIDIA.

Retrieval mixture-of-experts architecture

The latest Andromeda brings LLM-inspired mixture-of-experts (MoE) into retrieval through three architectural changes. On the user side, a single user tower becomes an ensemble of specialized user towers acting as experts. These draw on sequential ad engagement events, user-cohort representations, LLM-derived semantic features and cross-surface co-engagement signals.

In the candidate ads tower, a joint encoding strategy incorporates real-time serving context. In the interaction layer, dense neural networks give way to an MoE attention-based gating network that routes candidates across the most relevant user experts. Per Meta's figure description, only the top-K most relevant experts are activated to produce the user-ad relevance score.

Meta reports that model performance approaches ranking level at approximately 300x fewer inference-time FLOPs than the 2024 retrieval model. Model inference throughput has reached over 10 billion ads per second on a single GH200 GPU, which Meta says leaves room to add features, experts and depth without a proportional rise in GPU cost.

One fused GPU pipeline

The 2026 system pushes co-design further with two changes: a single GPU kernel that fuses model and system workloads for each user-ad interaction, and a strategy to co-locate per-request logic with the per-user-ad kernels in a single GPU box.

Meta explains that retrieval is memory-bandwidth bound. At billions of ads per second, even a small intermediate tensor per ad multiplies into terabytes per second of high bandwidth memory traffic. The fused kernel runs the whole per user-ad computation in one pass, keeping intermediate results in GPU registers and thread block shared memory instead of writing them to HBM. The 2024 launch fused only operations within model execution. The roughly 300x FLOPs reduction gave the headroom to include non-model system work too, and the fused kernel now runs at over 10 billion ads per second on a single GH200, according to Meta.

The co-location strategy targets a separate problem. Per-request operations such as index matching, which enforces advertisers' targeting rules, and top-K selection ran on separate local CPU hosts and communicated with the GPU scoring service over the network. At multi-billion ads per second, Meta says this traffic alone saturates the network link.

What it means for advertisers

Based on Meta's account, advertisers do not need to change anything to benefit. The retrieval system can consider a larger pool of ads, including the surge of AI-generated creatives, within the same latency budget. Meta also positions the efficiency gains as headroom for further model growth, which it says translates into greater advertiser value with each increment. The figures in the post are Meta's own claims from its production system.