Netflix Builds Automated Tool to Crop Videos for Vertical Feeds

Why Netflix needs vertical video at scale

Netflix says vertical formats are increasingly important to the company. It cites 1.2B followers across its social media accounts and the recent rollout of Clips, its vertical video feed in the Netflix mobile app.

Until now, creating vertical assets meant editors and producers manually reframed horizontal videos in editing tools. Netflix says that process does not scale as its catalog and use cases grow. The new system is designed to support both the in-app feed and external marketing campaigns with minimal manual effort.

The company argues that turning cinematic 16:9 footage into mobile-friendly formats takes more than a simple center crop. Because filmmaking often involves dynamic character blocking and complex camera movement, Netflix wants a "virtual camera" that follows the key action and produces crops that feel as intentional as the original shot.

A three-stage pipeline

Netflix frames the task as a tracking and temporal optimization problem. Its pipeline processes each asset in three stages and produces 9:16, 4:5, and 1:1 crops in a single pass, without additional computational overhead, according to the company.

• Shot detection and splitting: A neural-network-based shot boundary detector segments the video into atomic shots. For shots where several characters talk on opposite sides of the frame, an Active Speaker Detection approach splits a continuous shot into sub-shots, so the virtual camera can cut to different speakers. This detection uses an internal face detector and LipFlap, and Netflix says it works for both live-action and animated titles.
• Focus region discovery: The system combines face detections with visual saliency maps to prioritize which faces should stay in frame. In scenes with multiple characters, faces are ranked using signals such as active speaker status and relative size. The model combines these cues with shot-level content understanding to establish a continuous focus region.
• Temporal reframing: Tracking the focus region frame by frame would look shaky, so Netflix applies heuristics based on how the region moves during a shot. When displacement is small, the crop window stays static. When there is significant motion, the system computes a dynamic tracking window and smooths its path so panning feels intentional and cinematic.

Integration into Clips tooling and Premiere export

The resizing algorithm sits inside Netflix's internal Clips tool, so editors and producers can use it within their existing workflow. They can automatically generate multiple aspect-ratio versions of a clip with smart reframing. A unified in-app module offers shot-reframing controls, letting users review the algorithm's crops and make targeted adjustments.

For more complex shots or custom campaigns, producers can export the resizing parameters as XML and load them into external timeline editors such as Adobe Premiere. Netflix says this lets agencies and external partners start from the algorithm's framing decisions rather than recreating crops manually or relying on tools like Premiere's Auto Reframe. According to the company, its algorithm outperformed Auto Reframe in an internal test on a curated evaluation set.

What it means for Netflix teams and partners

Netflix says that combining automated resizing with manual refinement in one place significantly reduces production time and manual effort while preserving creative control. The company says it has reduced the time teams spend on repetitive resizing tasks and enabled faster experimentation across mobile feeds and marketing campaigns.

The blog post does not give figures for time saved or details of the evaluation set, so the performance claims remain Netflix's own. Netflix adds that it plans to extend the capabilities to new formats and workflows, with the stated goal of giving creatives more time to tell stories and delivering a consistently high-quality viewing experience for members and fans.