LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG's compatibility with alternative solutions such as retrieval-augmented generation (RAG).
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Knowledgeable-r1: Policy Optimization for Knowledge Exploration in Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) is a mainstream method for improving performance on knowledge-intensive tasks. However,current RAG systems often place too much emphasis on retrieved contexts. This can lead to relian…
counterfactualRAGRetrievalRetrieval-augmented GenerationKnowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks
Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks that require a compound understanding of knowledge. However, deployment of the LLMs in real-world applications can be c…
MedQAMemorizationStrategyQADuetRAG: Collaborative Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) methods augment the input of Large Language Models (LLMs) with relevant retrieved passages, reducing factual errors in knowledge-intensive tasks. However, contemporary RAG approaches …
PhilosophyRAGRetrievalRetrieval-augmented GenerationSimilarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts
In recent years, large language models (LLMs) have made remarkable achievements in various domains. However, the untimeliness and cost of knowledge updates coupled with hallucination issues of LLMs have curtailed their a…
AllHallucinationRAGRetrieval+1Reliable, Adaptable, and Attributable Language Models with Retrieval
Parametric language models (LMs), which are trained on vast amounts of web data, exhibit remarkable flexibility and capability. However, they still face practical challenges such as hallucinations, difficulty in adapting…
Question AnsweringRetrieval