KOEN
Read Original
#AI

Scaling Conditional Learned Retrieval for Pinterest Home Feed

Scaling Conditional Learned Retrieval for Pinterest Home Feed
01

Summary

Decoding Multi-Intent: How Pinterest Scaled Conditional Learned Retrieval for Millions of Users

An inside look at cutting P90 latency by 85% while unlocking massive personalized recommendation scale

This article details Pinterest's evolutionary journey in scaling Conditional Learned Retrieval (CLR) to solve the multi-intent retrieval challenge in home feeds. It explains how they unified multiple heuristic generators into a single, cohesive framework, integrated deep sequence modeling through foundation models, and pioneered lossless training and GPU-serving optimizations for production scale.

  • 01Transitioned from single-embedding retrieval to context-aware multi-embedding generation with CLR
  • 02Unified heterogeneous condition types (Interests, Pins, Boards) into a single, maintainable model
  • 03Adopted hierarchical Semantic IDs and residual-quantized VAEs to tackle cold-start issues for long-tail Pins
  • 04Achieved a 2x throughput boost via Request-level training deduplication and M-Falcon attention masking
  • 05Slashed P90 latency by 85% (80ms to 12ms) and achieved 7-figure cost savings via GPU serving and NVEmbed

RECOMMENDATION

Highly recommended for ML and recommendation system engineers struggling with serving costs, system complexity, and multi-interest representation in high-traffic production environments.

The Problem

Pinterest's home feed candidate generation is a large-scale User-to-Pin retrieval problem, but traditional two-tower models produce a single user embedding that struggles to capture diverse, simultaneous user interests like home renovation and recipes.

The Solution

The team scaled Conditional Learned Retrieval (CLR) by conditioning the user tower on explicit contexts (interests, Pins, Boards) within a unified model, integrating a Conditioned User Sequence Transformer and PinFM foundation model, and optimizing training and serving with M-Falcon attention masking and GPU-based NVEmbed serving.

The Result

The optimized framework achieved 7-figure financial cost savings, slashed P90 model latency by 85% (from 80ms to 12ms), delivered massive engagement metric wins, and successfully deprecated legacy heuristic candidate generators.

Trade-off

Integrating large foundation models significantly increased computational complexity, requiring the engineering team to design and maintain custom infrastructure optimizations, such as M-Falcon block attention masks and request-level deduplication pipelines to control high GPU overhead.

03

Key Concepts

Concept · 01

Conditional Learned Retrieval (CLR)

An advanced retrieval paradigm that extends the traditional two-tower model by conditioning the user tower on specific explicit contexts, outputting multiple dynamic, context-aware embeddings instead of a single static one.

  • Applied in Pinterest's home feed to generate highly customized candidate sets based on dynamically changing contexts like user interests, specific Pins, or Boards.
Concept · 02

M-Falcon Optimization

A lossless training optimization technique that exploits causal attention to append all target conditions for a given user to a single user sequence, preventing redundant sequence computations in a batch.

  • Utilized a custom block attention mask to isolate flattened condition tokens from each other, dramatically decreasing the effective batch size and GPU memory footprint during Transformer forward passes.
Concept · 03

Semantic IDs

Hierarchical codes generated by quantizing static content embeddings using residual-quantized VAEs, allowing visually and semantically similar items to share adjacent regions in the ID embedding space.

  • Replaced a massive 20GB random hash ID table with five small hierarchical embedding tables, enabling long-tail items with sparse engagement history to inherit collaborative signals from popular items.