Training Personalized Recommendation Systems from (GPU) Scratch: Look Forward not Backwards
Personalized recommendation models (RecSys) are one of the most popular machine learning workload serviced by hyperscalers. A critical challenge of training RecSys is its high memory capacity requirements, reaching hundreds of GBs to TBs of model size. In RecSys, the so-called embedding layers account for the majority of memory usage so current systems employ a hybrid CPU-GPU design to have the large CPU memory store the memory hungry embedding layers. Unfortunately, training embeddings involve several memory bandwidth intensive operations which is at odds with the slow CPU memory, causing performance overheads. Prior work proposed to cache frequently accessed embeddings inside GPU memory as means to filter down the embedding layer traffic to CPU memory, but this paper observes several limitations with such cache design. In this work, we present a fundamentally different approach in designing embedding caches for RecSys. Our proposed ScratchPipe architecture utilizes unique properties of RecSys training to develop an embedding cache that not only sees the past but also the "future" cache accesses. ScratchPipe exploits such property to guarantee that the active working set of embedding layers can "always" be captured inside our proposed cache design, enabling embedding layer training to be conducted at GPU memory speed.
Code (0)
등록된 구현이 없습니다.
Tasks
CPUGPURecommendation SystemsSimilar Papers 제목 키워드 기반
MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions
Deep neural networks are widely used in personalized recommendation systems. Unlike regular DNN inference workloads, recommendation inference is memory-bound due to the many random memory accesses needed to lookup the em…
CPURecommendation SystemsPersonalized Re-ranking for Improving Diversity in Live Recommender Systems
Users of industrial recommender systems are normally suggesteda list of items at one time. Ideally, such list-wise recommendationshould provide diverse and relevant options to the users. However, in practice, list-wise r…
DiversityRecommendation SystemsRe-RankingCapturing Popularity Trends: A Simplistic Non-Personalized Approach for Enhanced Item Recommendation
Recommender systems have been gaining increasing research attention over the years. Most existing recommendation methods focus on capturing users' personalized preferences through historical user-item interactions, which…
Decision MakingRecommendation SystemsGraph Neural Networks for Recommendation: Reproducibility, Graph Topology, and Node Representation
Graph neural networks (GNNs) have gained prominence in recommendation systems in recent years. By representing the user-item matrix as a bipartite and undirected graph, GNNs have demonstrated their potential to capture s…
Graph LearningRecommendation SystemsLarge Language Models are Learnable Planners for Long-Term Recommendation
Planning for both immediate and long-term benefits becomes increasingly important in recommendation. Existing methods apply Reinforcement Learning (RL) to learn planning capacity by maximizing cumulative reward for long-…
Decision MakingLanguage ModellingLarge Language ModelReinforcement Learning (RL)