RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation
Personalized image generation is crucial for improving the user experience, as it renders reference images into preferred ones according to user visual preferences. Although effective, existing methods face two main issues. First, existing methods treat all items in the user historical sequence equally when extracting user preferences, overlooking the varying semantic similarities between historical items and the reference item. Disproportionately high weights for low-similarity items distort users' visual preferences for the reference item. Second, existing methods heavily rely on consistency between generated and reference images to optimize the generation, which leads to underfitting user preferences and hinders personalization. To address these issues, we propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR). Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item, thereby extracting more refined users' visual preferences for the reference item. Then we introduce a novel rank task based on the multi-modal ranking model to optimize the personalization of the generated images instead of forcing depend on consistency. Extensive experiments and human evaluations on three real-world datasets demonstrate that RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationPersonalized Image GenerationRetrievalSimilar Papers 제목 키워드 기반
RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models
The escalating challenge of misinformation, particularly in political discourse, requires advanced fact-checking solutions; this is even clearer in the more complex scenario of multimodal claims. We tackle this issue usi…
Fact CheckingLanguage ModelingLanguage ModellingLarge Language Model+7ClusterRAG: Cluster-Based Collaborative Filtering for Personalized Retrieval-Augmented Generation
Personalized Retrieval-Augmented Generation (RAG) relies on accurately selecting user-relevant documents. In practice, existing RAG approaches often suffer from high retrieval costs and overlook that collaborative signal…
Collaborative FilteringPersonalized Graph-Based Retrieval for Large Language Models
As large language models (LLMs) evolve, their ability to deliver personalized and context-aware responses offers transformative potential for improving user experiences. Existing personalization approaches, however, ofte…
Knowledge GraphsRetrievalRetrieval-augmented GenerationText GenerationOptimization Methods for Personalizing Large Language Models through Retrieval Augmentation
This paper studies retrieval-augmented approaches for personalizing large language models (LLMs), which potentially have a substantial impact on various applications and domains. We propose the first attempt to optimize …
Knowledge DistillationLanguage ModelingLanguage ModellingRetrievalRetrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation
Recently, the personalization of Large Language Models (LLMs) to generate content that aligns with individual user preferences has garnered widespread attention. Personalized Retrieval-Augmented Generation (RAG), which r…
Collaborative FilteringContrastive LearningRAGRecommendation Systems+4