RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation
Large language models (LLMs) fine-tuned for text-retrieval have demonstrated state-of-the-art results across several information retrieval (IR) benchmarks. However, supervised training for improving these models requires numerous labeled examples, which are generally unavailable or expensive to acquire. In this work, we explore the effectiveness of extending reverse engineered adaptation to the context of information retrieval (RE-AdaptIR). We use RE-AdaptIR to improve LLM-based IR models using only unlabeled data. We demonstrate improved performance both in training domains as well as zero-shot in domains where the models have seen no queries. We analyze performance changes in various fine-tuning scenarios and offer findings of immediate use to practitioners.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalRetrievalText RetrievalSimilar Papers 제목 키워드 기반
RE-Adapt: Reverse Engineered Adaptation of Large Language Models
We introduce RE-Adapt, an approach to fine-tuning large language models on new domains without degrading any pre-existing instruction-tuning. We reverse engineer an adapter which isolates what an instruction-tuned model …
Instruction FollowingRetrievalRetrieval-augmented GenerationClassification of Reverse-Engineered Class Diagram and Forward-Engineered Class Diagram using Machine Learning
UML Class diagram is very important to visualize the whole software we are working on and helps understand the whole system in the easiest way possible by showing the system classes, its attributes, methods, and relation…
BIG-bench Machine LearningGeneral ClassificationParameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts
Designing single-task image restoration models for specific degradation has seen great success in recent years. To achieve generalized image restoration, all-in-one methods have recently been proposed and shown potential…
DenoisingDiversityImage DenoisingImage Restoration+2Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
Recent research demonstrates the effectiveness of using fine-tuned language models~(LM) for dense retrieval. However, dense retrievers are hard to train, typically requiring heavily engineered fine-tuning pipelines to re…
Language ModelingLanguage ModellingPassage RetrievalRetrievalThe Quantum Imitation Game: Reverse Engineering of Quantum Machine Learning Models
Quantum Machine Learning (QML) amalgamates quantum computing paradigms with machine learning models, providing significant prospects for solving complex problems. However, with the expansion of numerous third-party vendo…
Quantum Machine Learning