BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain

2024-09-30 · Kaisi Guan, Qian Cao, Yuchong Sun, Xiting Wang, Ruihua Song

Retrieval Augmented Generation (RAG) system is important in domains such as e-commerce, which has many long-tail entities and frequently updated information. Most existing works adopt separate modules for retrieval and generation, which may be suboptimal since the retrieval task and the generation task cannot benefit from each other to improve performance. We propose a novel Backbone Shared RAG framework (BSharedRAG). It first uses a domain-specific corpus to continually pre-train a base model as a domain-specific backbone model and then trains two plug-and-play Low-Rank Adaptation (LoRA) modules based on the shared backbone to minimize retrieval and generation losses respectively. Experimental results indicate that our proposed BSharedRAG outperforms baseline models by 5% and 13% in Hit@3 upon two datasets in retrieval evaluation and by 23% in terms of BLEU-3 in generation evaluation. Our codes, models, and dataset are available at https://bsharedrag.github.io.

📄 PDF Abstract BibTeX arXiv:2409.20075

Code (0)

등록된 구현이 없습니다.

Tasks

RAG RetrievalRetrieval-augmented Generation

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

WordPiece 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Weight Decay 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

BERT BERT, or Bidirectional Encoder Representations from Transformers, improves upon standard Transformers by removing the…

In-Context Optimization for Retrieval-Augmented Generation: A Gradient-Descent Perspective

2026-05-25 · Mingchen Li, Jiatan Huang, Chuxu Zhang, Liang Zhao 외 arxiv

In-context learning has recently been linked to implicit gradient descent in linear self-attention models, suggesting that context can induce a forward-pass update. Retrieval-augmented generation (RAG) also relies on con…

Findings of the MAGMaR 2026 Shared Task

2026-06-10 · Alexander Martin, Dengjia Zhang, Joel Brogan, Francis Ferraro 외 arxiv

This overview paper presents the results of the shared task for the second workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR). In this shared task participants submitted systems focused on eith…

Video Retrieval

Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answering

2025-09-28 · Muhammad Abu Ahmad, Mohamad Ballout, Raia Abu Ahmad, Elia Bruni arxiv

This paper presents our submission to the QIAS 2025 shared task on Islamic knowledge understanding and reasoning. We developed a hybrid retrieval-augmented generation (RAG) system that combines sparse and dense retrieval…

Question Answering

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

2026-06-27 · Ansh Kamthan arxiv

Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty…

Question Answering

RAGViz: Diagnose and Visualize Retrieval-Augmented Generation

2024-11-04 · Tevin Wang, Jingyuan He, Chenyan Xiong

Retrieval-augmented generation (RAG) combines knowledge from domain-specific sources into large language models to ground answer generation. Current RAG systems lack customizable visibility on the context documents and t…

Answer Generation GPU Language Modeling Language Modelling+4