ROCS Boosts Efficiency for Large-Scale Recommendation Systems.

Modern recommendation systems achieve high prediction quality by employing complex models with extensive feature-interaction and sequence modules. However, the computational cost of evaluating these models for numerous candidates per user request often limits their scalability in production environments. Researchers have introduced Request-Oriented Compute Sharing (ROCS), a novel modeling and inference paradigm designed to address this efficiency challenge. ROCS optimizes recommendation inference by delaying request-candidate interactions and isolating candidate-dependent representations, allowing substantial parts of the model to be evaluated only once per user request, rather than for every single candidate. To implement ROCS, the team developed Generalized Layer Masking (GLM) for feature-interaction architectures and Deep Cross Attention (DCA) for sequence architectures, alongside In-Kernel Broadcast Optimization (IKBO) for efficient GPU deployment. Experiments show ROCS improves quality-efficiency tradeoffs, delivering up to a 3x query-per-second (QPS) increase for retrieval models and a 50% QPS gain with improved quality for ranking models, with successful deployment in large-scale production systems.