Netflix Details Three Ways to Plug Its Foundation Model Into Personalization

Why Netflix centralized preference learning

The Netflix homepage is powered by several specialized models, and maintaining and improving each one takes significant time and resources. Netflix therefore decided to centralize member preference learning in one large model trained on comprehensive user interaction histories and content data, which then passes its learnings on to other models.

The authors, Divya Gadde, Ko-Jen Hsiao, Dhaval Patel and Moumita Bhattacharya, say there is plenty of literature on training and running large transformer-based models, but little practical guidance on how to integrate such models into existing production systems. Different teams at Netflix have different latency requirements, tech stack constraints and levels of appetite for using the full power of a large model, which is why the company tried several approaches.

Embeddings: the low-cost starting point

The first approach uses embeddings. Netflix takes the hidden state of the last user event as the profile embedding, and the weights from the item tower as item embeddings. The refresh cycle is elaborate: the Foundation Model is pre-trained from scratch every month, then fine-tuned every day on the latest data. The daily fine-tuning also expands the entity id space to include newly launching titles. Batch inference then refreshes the profile and item embeddings and publishes them to the Embedding Store.

A key part of this setup is an embedding stabilization technique. Retraining from scratch produces completely different embedding spaces between runs because of random initialization, and embeddings also drift during daily fine-tuning. Mapping each day's embeddings into the same space lets downstream models consume pre-computed embeddings as features.

The Embedding Store is a feature store for embeddings built by Netflix's platform team. It handles versioning and timestamping and offers offline and online access. According to Netflix, this is often the initial and most straightforward route, with a low barrier to entry and a relatively small impact on training and inference costs. The drawback is staleness: the gap between embedding computation and downstream inference can hurt freshness, especially for use cases that need real-time adaptability.

Netflix says it considers embeddings a low-cost, high-leverage option and has stepped up investment in a near-realtime embedding generation framework. That framework can update embeddings based on user actions during a session, though it cannot be used for very large models.

Subgraph and fine-tuning for deeper integration

In the subgraph approach, the Foundation Model decoder stack becomes part of the application model's full graph. It processes raw user interaction sequences and feeds the resulting representations into the downstream model. Applications can fine-tune the subgraph as part of their own training, and there is no time gap between Foundation Model inference and application model inference. Teams can also use specific layers not exposed through the Embedding Store.

The costs are real. Application models must generate all the features the subgraph needs, which adds time, compute and complexity to feature generation. Merging the subgraph also increases model size and inference time. The Foundation Model team provides reusable code and jobs to make feature generation more efficient, and splits the subgraph so it runs once per profile per request, shared by all items in the request. Netflix would reserve this approach for high impact use cases where metric gains justify the cost and complexity.

The third approach is fine-tuning the Foundation Model and using it directly. The model is trained on a next-token prediction objective, with user interactions as tokens. Different surfaces value interactions differently: for the 'Trending now' row, recent interactions on trending titles may matter more than old ones. Application teams can do full parameter fine-tuning or freeze some layers, and can add a different output head with a different objective. Netflix built a fine-tuning framework so teams can develop their own custom versions of the model.

What it means for Netflix's teams

The post frames the three options as a spectrum. Embeddings offer an easy entry point with some staleness. The subgraph delivers fresher, deeper integration at higher complexity and compute cost. Fine-tuning lets teams tailor the model to a specific product surface. Which one fits depends on each team's latency needs, technical constraints and the value they expect from the full model.