Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation

2024-04-09 · Alireza Salemi, Surya Kallumadi, Hamed Zamani

This paper studies retrieval-augmented approaches for personalizing large language models (LLMs), which potentially have a substantial impact on various applications and domains. We propose the first attempt to optimize the retrieval models that deliver a limited number of personal documents to large language models for the purpose of personalized generation. We develop two optimization algorithms that solicit feedback from the downstream personalized generation tasks for retrieval optimization--one based on reinforcement learning whose reward function is defined using any arbitrary metric for personalized generation and another based on knowledge distillation from the downstream LLM to the retrieval model. This paper also introduces a pre- and post-generation retriever selection model that decides what retriever to choose for each LLM input. Extensive experiments on diverse tasks from the language model personalization (LaMP) benchmark reveal statistically significant improvements in six out of seven datasets.

📄 PDF Abstract BibTeX arXiv:2404.05970

Code (1)

lamp-benchmark/lamp 공식 구현 pytorch

Tasks

Knowledge DistillationLanguage ModelingLanguage Modelling Retrieval

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

CoPL: Collaborative Preference Learning for Personalizing LLMs

2025-03-03 · Youngbin Choi, Seunghyuk Cho, Minjong Lee, Moonjeong Park 외

Personalizing large language models (LLMs) is important for aligning outputs with diverse user preferences, yet existing methods struggle with flexibility and generalization. We propose CoPL (Collaborative Preference Lea…

Collaborative Filtering

Evaluating Approaches to Personalizing Language Models

2020-05-01 · LREC 2020 5 · Milton King, Paul Cook

In this work, we consider the problem of personalizing language models, that is, building language models that are tailored to the writing style of an individual. Because training language models requires a large amount …

Language Modeling Language Modelling

Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models

2024-09-14 · Alireza Salemi, Hamed Zamani

Privacy-preserving methods for personalizing large language models (LLMs) are relatively under-explored. There are two schools of thought on this topic: (1) generating personalized outputs by personalizing the input prom…

parameter-efficient fine-tuning Privacy Preserving RAG Retrieval

JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew

2026-04-20 · Itay Razumenko, Arnon Sturm, Nir Grinberg arxiv

Despite significant advances in large language models, personalizing them for individual decision-makers remains an open problem. Here, we introduce a synthetic-organic supervision pipeline that transforms raw judicial d…

parameter-efficient fine-tuning Semantic Similarity

Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities

2025-07-02 · Yingqiang Gao, Kaede Johnson, David Froehlich, Luisa Carrer 외 arxiv

Automatic text simplification (ATS) aims to enhance language accessibility for various target groups, particularly persons with intellectual disabilities. Recent advancements in generative AI, especially large language m…

Text Simplification