paper-with-me

홈 › Papers

CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems

2025-06-24 · Haochen Zhang, Tianyi Zhang, Junze Yin, Oren Gal, Anshumali Shrivastava, Vladimir Braverman

Recommender systems play a pivotal role in providing relevant content to users. With the rapid development of large language models (LLMs), researchers have begun utilizing LLMs to build more powerful recommender systems. However, existing approaches that focus on aligning LLMs with recommendation tasks do not fully leverage their sequential information processing capabilities, leading to suboptimal performance. In this paper, we propose a novel system called compressed vocabulary expansion (CoVE). In CoVE, each item is assigned a unique ID within the expanded vocabulary. Our framework effectively capitalizes on sequence understanding abilities of LLMs, significantly enhancing their performance on recommendation tasks. Additionally, we compress the embedding layer, making CoVE practical for large-scale industrial applications. The effectiveness and performance of CoVE are demonstrated through comprehensive experiments on multiple recommendation datasets and comparisons with prior works. Our code can be found at https://github.com/HaochenZhang717/CoVE-official-Repo.

📄 PDF Abstract BibTeX arXiv:2506.19993

Code (1)

haochenzhang717/cove-official-repo 공식 구현 pytorch

Tasks

Recommendation Systems

Methods 이 논문이 사용한 방법론

GloVe GloVe Embeddings are a type of word embedding that encode the co-occurrence probability ratio between two words as vector differences. GloVe uses a weighted least squares…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…
Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…
BiLSTM A Bidirectional LSTM, or biLSTM, is a sequence processing model that consists of two LSTMs: one taking the input in a forward direction, and the other in a backwards…
CoVe CoVe, or Contextualized Word Vectors, uses a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine…
Focus 설명 없음

Similar Papers 제목 키워드 기반

Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs

2024-10-02 · Chengyuan Liu, Shihang Wang, Lizhi Qing, Kun Kuang 외

While Large Language Models (LLMs) demonstrate impressive generation abilities, they frequently struggle when it comes to specialized domains due to their limited domain-specific knowledge. Studies on domain-specific LLM…

Now It Sounds Like You: Learning Personalized Vocabulary On Device

2023-05-05 · Sid Wang, Ashish Shenoy, Pierce Chuang, John Nguyen

In recent years, Federated Learning (FL) has shown significant advancements in its ability to perform various natural language processing (NLP) tasks. This work focuses on applying personalized FL for on-device language …

Federated LearningLanguage ModelingLanguage Modelling

How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?

2024-06-17 · Atsuki Yamaguchi, Aline Villavicencio, Nikolaos Aletras

Large language models (LLMs) have shown remarkable capabilities in many languages beyond English. Yet, LLMs require more inference steps when generating non-English text due to their reliance on English-centric tokenizer…

Cross-Lingual Transfer

LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

2026-05-07 · Roy Xie, Dan Friedman, Donghan Yu, Bowen Pan 외 arxiv

Vision Language Models (VLMs) offer the exciting possibility of processing text as rendered images, bypassing the need for tokenizing the text into long token sequences. Since VLM image encoders map fixed-size images to …

A Fast Quantum Image Compression Algorithm based on Taylor Expansion

2025-02-15 · Vu Tuan Hai, Huynh Ho Thi Mong Trinh, Pham Hoai Luan

With the increasing demand for storing images, traditional image compression methods face challenges in balancing the compressed size and image quality. However, the hybrid quantum-classical model can recover this weakne…

Image Compression