ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation System
The model size growth of personalized recommendation systems poses new challenges for inference. Weight-sharing algorithms have been proposed for size reduction, but they increase memory access. Recent advancements in processing-in-memory (PIM) enhanced the model throughput by exploiting memory parallelism, but such algorithms introduce massive CPU-PIM communication into prior PIM systems. We propose ProactivePIM, a PIM system for weight-sharing recommendation system acceleration. ProactivePIM integrates a cache within the PIM with a prefetching scheme to leverage a unique locality of the algorithm and eliminate communication overhead through a subtable mapping strategy. ProactivePIM achieves a 4.8x speedup compared to prior works.
Code (0)
등록된 구현이 없습니다.
Tasks
CPURecommendation SystemsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
Tying the weights of the target word embeddings with the target word classifiers of neural machine translation models leads to faster training and often to better translation quality. Given the success of this parameter …
DecoderMachine TranslationTranslationWord EmbeddingsLayer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency
Transformer-based self-supervised models are trained as feature extractors and have empowered many downstream speech tasks to achieve state-of-the-art performance. However, both the training and inference process of thes…
speech-recognitionSpeech RecognitionSubformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers
Transformers have shown improved performance when compared to previous architectures for sequence processing such as RNNs. Despite their sizeable performance gains, as recently suggested, the model is computationally exp…
Abstractive Text SummarizationLanguage ModelingLanguage ModellingMachine Translation+1Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
Neural Architecture Search (NAS) has shown great potentials in finding better neural network designs. Sample-based NAS is the most reliable approach which aims at exploring the search space and evaluating the most promis…
Bayesian OptimizationNeural Architecture SearchWeight Tying Biases Token Embeddings Towards the Output Space
Weight tying, i.e. sharing parameters between input and output embedding matrices, is common practice in language model design, yet its impact on the learned embedding space remains poorly understood. In this paper, we s…