KOEN
Read Original
#AI

Beyond Two Towers: Launching the 3-Tower Engagement Co-Train Model (Part 2)

Beyond Two Towers: Launching the 3-Tower Engagement Co-Train Model (Part 2)
01

Summary

Beyond Two Towers: Pinterest's Highly Scalable 3-Tower Real-Time Ranking Model

Deploying a hybrid 3-tower architecture at Pinterest scale by balancing speed and model precision

This article highlights Pinterest's transition beyond classic two-tower models by launching a 3-tower engagement co-train model for standard ads. It explains how they successfully engineered a high-performance recommendation system using custom-tailored cross-attention layers, system optimizations, and a smart two-stage scoring pipeline to meet strict latency requirements.

  • 01Introduces a third 'cross tower' to capture interactions between user sequence features and candidate ads under a unified multi-task model.
  • 02Caches complex candidate feature preprocessing offline in the Pin tower, saving about 2ms of serving-time latency at 100K candidates.
  • 03Reduces inference precision from FP32 to BF16, slashing P99 model latency by roughly 50% without degrading quality.
  • 04Leverages a two-stage scoring flow where all candidates are scored by a fast 2-tower model and only the top 40K are evaluated with the 3-tower model.
  • 05Delivered ~1% gains in online CTR, gCTR30, and oCTR with minimal increase in infrastructure costs.

RECOMMENDATION

Highly recommended for machine learning engineers and recommendation system developers seeking practical methods to deploy highly expressive neural networks in heavy traffic and tight latency budget environments.

The Problem

Pinterest's lightweight ranking stage must score hundreds of thousands of candidates (up to 200K at P99) per request, making it extremely difficult to deploy complex, expressive machine learning models under strict real-time serving latency and cost constraints.

The Solution

They designed a 3-tower co-train model that extends the existing two-tower (Query, Pin) architecture with a third 'cross tower' for user-sequence and candidate interactions, and deployed it inside a two-stage scoring flow that scales by using fast 2-tower scoring for all candidates and rich 3-tower scoring for a top-K subset.

The Result

The model achieved a ~30% reduction in offline loss for three-tower predictions across engagement tasks, and extensive serving optimizations (e.g., BF16 precision, moving LayerNorm) reduced P99 latency from over 200ms to ~30ms (only 1-2ms slower than production). This translated into online lifts of ~1% in CTR, gCTR30, and oCTR, along with CPC reductions.

Trade-off

More complex attention models and MMoE layers had to be simplified to a candidate-derived query token attention and a shared-bottom MLP to stay within latency budgets, and the heuristic approach of blending 2-tower and 3-tower scores for the final selection could theoretically introduce bias toward items that already scored high in the first stage.

03

Key Concepts

Concept · 01

3-Tower Co-train Model

An architecture that extends the traditional query and item (Pin) two-tower design with a third 'cross tower' dedicated to processing user interaction history, trained via multi-task loss.

  • It simultaneously outputs fast two-tower dot-product scores and highly accurate three-tower scores within a single unified model.
  • The model uses a weighted sum of main three-tower losses and auxiliary two-tower losses during training.
Concept · 02

Cross Tower & Attention

A structural module utilizing cross-attention over on-site user behavior sequences and candidate features to capture high-order interaction patterns.

  • To control transformer compute, candidate features are used to generate query tokens which then interact with user sequences for attention calculation.
Concept · 03

Two-Stage Scoring Flow

A serving pipeline designed to score a large candidate pool quickly with a fast model first, followed by a deeper, more complex evaluation on a selected top-K subset.

  • In stage one, the 2-tower co-train head filters the full candidate set to select a 'utility top-K' of 40K candidates.
  • In stage two, only the top 40K subset is rescored using the computationally expensive 3-tower model.