CompAct: Compressing Retrieved Documents Actively for Question Answering

2024-07-12 · Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, Jaewoo Kang

Retrieval-augmented generation supports language models to strengthen their factual groundings by providing external contexts. However, language models often face challenges when given extensive information, diminishing their effectiveness in solving questions. Context compression tackles this issue by filtering out irrelevant information, but current methods still struggle in realistic scenarios where crucial information cannot be captured with a single-step approach. To overcome this limitation, we introduce CompAct, a novel framework that employs an active strategy to condense extensive documents without losing key information. Our experiments demonstrate that CompAct brings significant improvements in both performance and compression rate on multi-hop question-answering benchmarks. CompAct flexibly operates as a cost-efficient plug-in module with various off-the-shelf retrievers or readers, achieving exceptionally high compression rates (47x).

📄 PDF Abstract BibTeX arXiv:2407.09014

Code (1)

dmis-lab/compact 공식 구현 pytorch

Tasks

Multi-hop Question Answering Question Answering RetrievalRetrieval-augmented Generation

RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation

2023-10-06 · Fangyuan Xu, Weijia Shi, Eunsol Choi

Retrieving documents and prepending them in-context at inference time improves performance of language model (LMs) on a wide range of tasks. However, these documents, often spanning hundreds of words, make inference subs…

Language Modeling Language Modelling Open-Domain Question Answering Question Answering+1

Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation

2024-05-06 · Kaize Shi, Xueyao Sun, Qing Li, Guandong Xu

Large Language Models (LLMs) have made significant strides in information acquisition. However, their overreliance on potentially flawed parametric knowledge leads to hallucinations and inaccuracies, particularly when ha…

Abstract Meaning Representation Open-Domain Question Answering Question Answering RAG+2

BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression

2024-10-20 · Yuankai Li, Jia-Chen Gu, Di wu, Kai-Wei Chang 외

Retrieval-augmented generation (RAG) can supplement large language models (LLMs) by integrating external knowledge. However, as the number of retrieved documents increases, the input length to LLMs grows linearly, causin…

In-Context Learning Long-Context Understanding Open-Domain Question Answering Question Answering+3

Iterative Hierarchical Attention for Answering Complex Questions over Long Documents

2021-06-01 · Haitian Sun, William W. Cohen, Ruslan Salakhutdinov

We propose a new model, DocHopper, that iteratively attends to different parts of long, hierarchically structured documents to answer complex questions. Similar to multi-hop question-answering (QA) systems, at each step,…

Multi-hop Question Answering Question Answering Sentence Embedding

Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA

2025-05-22 · Rishabh Maheshwary, Masoud Hashemi, Khyati Mahajan, Shiva Krishna Reddy Malay 외

Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reason over retrieved content and limits per…

Multi-hop Question Answering Question Answering RAG