MST-R: Multi-Stage Tuning for Retrieval Systems and Metric Evaluation

2024-12-13 · Yash Malviya, Karan Dhingra, Maneesh Singh

Regulatory documents are rich in nuanced terminology and specialized semantics. FRAG systems: Frozen retrieval-augmented generators utilizing pre-trained (or, frozen) components face consequent challenges with both retriever and answering performance. We present a system that adapts the retriever performance to the target domain using a multi-stage tuning (MST) strategy. Our retrieval approach, called MST-R (a) first fine-tunes encoders used in vector stores using hard negative mining, (b) then uses a hybrid retriever, combining sparse and dense retrievers using reciprocal rank fusion, and then (c) adapts the cross-attention encoder by fine-tuning only the top-k retrieved results. We benchmark the system performance on the dataset released for the RIRAG challenge (as part of the RegNLP workshop at COLING 2025). We achieve significant performance gains obtaining a top rank on the RegNLP challenge leaderboard. We also show that a trivial answering approach games the RePASs metric outscoring all baselines and a pre-trained Llama model. Analyzing this anomaly, we present important takeaways for future research.

📄 PDF Abstract BibTeX arXiv:2412.10313

Code (1)

indic-aidias/mst-r 공식 구현 pytorch

Tasks

Retrieval

Methods 이 논문이 사용한 방법론

LLaMA LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were…

PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Models

2023-12-05 · Wei-Cheng Chang, Jyun-Yu Jiang, Jiong Zhang, Mutasem Al-Darabsah 외

Embedding-based Retrieval Models (ERMs) have emerged as a promising framework for large-scale text retrieval problems due to powerful large language models. Nevertheless, fine-tuning ERMs to reach state-of-the-art result…

Retrieval Text Retrieval

Invar-RAG: Invariant LLM-aligned Retrieval for Better Generation

2024-11-11 · Ziwei Liu, Liang Zhang, Qian Li, Jianghua Wu 외

Retrieval-augmented generation (RAG) has shown impressive capability in providing reliable answer predictions and addressing hallucination problems. A typical RAG implementation uses powerful retrieval models to extract …

Hallucination Information Retrieval Open-Domain Question Answering Question Answering+4

MMRAG-RFT: Two-stage Reinforcement Fine-tuning for Explainable Multi-modal Retrieval-augmented Generation

2025-12-19 · Shengwei Zhao, Jingwen Yao, Sitong Wei, Linhai Xu 외 arxiv

Multi-modal Retrieval-Augmented Generation (MMRAG) enables highly credible generation by integrating external multi-modal knowledge, thus demonstrating impressive performance in complex multi-modal scenarios. However, ex…

Reinforcement Learning Response Generation Answer Generation

FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval

2026-07-30 · Bohan Hou, Haoqiang Lin, Xuemeng Song, Haokun Wen 외 arxiv

Due to their strong generalizable multimodal processing and reasoning capabilities, Multimodal Large Language Models (MLLMs) have demonstrated significant potential as universal image retrievers, effectively addressing d…

Image Retrieval Visual Dialog

KnowPrefix-Tuning: A Two-Stage Prefix-Tuning Framework for Knowledge-Grounded Dialogue Generation

2023-06-27 · Jiaqi Bai, Zhao Yan, Jian Yang, Xinnian Liang 외

Existing knowledge-grounded conversation systems generate responses typically in a retrieve-then-generate manner. They require a large knowledge base and a strong knowledge retrieval component, which is time- and resourc…

Dialogue Generation Response Generation Retrieval