Retrieval augmentation of large language models for lay language generation
Recent lay language generation systems have used Transformer models trained on a parallel corpus to increase health information accessibility. However, the applicability of these models is constrained by the limited size and topical breadth of available corpora. We introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. The abstract and the corresponding lay language summary are written by domain experts, assuring the quality of our dataset. Furthermore, qualitative evaluation of expert-authored plain language summaries has revealed background explanation as a key strategy to increase accessibility. Such explanation is challenging for neural models to generate because it goes beyond simplification by adding content absent from the source. We derive two specialized paired corpora from CELLS to address key challenges in lay language generation: generating background explanations and simplifying the original abstract. We adopt retrieval-augmented models as an intuitive fit for the task of background explanation generation, and show improvements in summary quality and simplicity while maintaining factual correctness. Taken together, this work presents the first comprehensive study of background explanation for lay language generation, paving the path for disseminating scientific knowledge to a broader audience. CELLS is publicly available at: https://github.com/LinguisticAnomalies/pls_retrieval.
Code (1)
Tasks
Explanation GenerationRetrievalText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
LaMP: When Large Language Models Meet Personalization
This paper highlights the importance of personalization in large language models and introduces the LaMP benchmark -- a novel benchmark for training and evaluating language models for producing personalized outputs. LaMP…
Language ModelingLanguage ModellingNatural Language UnderstandingRetrieval+3Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation
This paper studies retrieval-augmented approaches for personalizing large language models (LLMs), which potentially have a substantial impact on various applications and domains. We propose the first attempt to optimize …
Knowledge DistillationLanguage ModelingLanguage ModellingRetrievalUniRAG: Universal Retrieval Augmentation for Large Vision Language Models
Recently, Large Vision Language Models (LVLMs) have unlocked many complex use cases that require Multi-Modal (MM) understanding (e.g., image captioning or visual question answering) and MM generation (e.g., text-guided i…
Image CaptioningImage GenerationQuestion AnsweringRetrieval+1Retrieval meets Long Context Large Language Models
Extending the context window of large language models (LLMs) is getting popular recently, while the solution of augmenting LLMs with retrieval has existed for years. The natural questions are: i) Retrieval-augmentation v…
16k4kFew-Shot LearningNatural Questions+2ZusammenQA: Data Augmentation with Specialized Models for Cross-lingual Open-retrieval Question Answering System
This paper introduces our proposed system for the MIA Shared Task on Cross-lingual Open-retrieval Question Answering (COQA). In this challenging scenario, given an input question the system has to gather evidence documen…
Answer GenerationData AugmentationLanguage ModellingPassage Retrieval+2