홈 › Papers

An Evaluation Framework for Attributed Information Retrieval using Large Language Models

2024-09-12 · Hanane Djeddal, Pierre Erbacher, Raouf Toukal, Laure Soulier, Karen Pinel-Sauvagnat, Sophia Katrenko, Lynda Tamine

With the growing success of Large Language models (LLMs) in information-seeking scenarios, search engines are now adopting generative approaches to provide answers along with in-line citations as attribution. While existing work focuses mainly on attributed question answering, in this paper, we target information-seeking scenarios which are often more challenging due to the open-ended nature of the queries and the size of the label space in terms of the diversity of candidate-attributed answers per query. We propose a reproducible framework to evaluate and benchmark attributed information seeking, using any backbone LLM, and different architectural designs: (1) Generate (2) Retrieve then Generate, and (3) Generate then Retrieve. Experiments using HAGRID, an attributed information-seeking dataset, show the impact of different scenarios on both the correctness and attributability of answers.

📄 PDF Abstract BibTeX arXiv:2409.08014

Code (1)

hanane-djeddal/attributed-ir 공식 구현 pytorch

Tasks

DiversityInformation Retrieval Question Answering Retrieval

ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generation

2025-02-14 · Shu Wang, Yixiang Fang, Yingli Zhou, Xilin Liu 외

Retrieval-Augmented Generation (RAG) has proven effective in integrating external knowledge into large language models (LLMs) for question-answer (QA) tasks. The state-of-the-art RAG approaches often use the graph data a…

RAG Retrieval Retrieval-augmented Generation

LLM-Powered Text-Attributed Graph Anomaly Detection via Retrieval-Augmented Reasoning

2025-11-16 · Haoyan Xu, Ruizhi Qian, Zhengtao Yao, Ziyi Liu 외 arxiv

Anomaly detection on attributed graphs plays an essential role in applications such as fraud detection, intrusion monitoring, and misinformation analysis. However, text-attributed graphs (TAGs), in which node information…

Graph Anomaly Detection Prompt Engineering Fraud Detection

HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution

2023-07-31 · Ehsan Kamalloo, Aref Jafari, Xinyu Zhang, Nandan Thakur 외

The rise of large language models (LLMs) had a transformative impact on search, ushering in a new era of search engines that are capable of generating search results in natural language text, imbued with citations for su…

Information Retrieval Informativeness Retrieval

Atomic Fact Decomposition Helps Attributed Question Answering

2024-10-22 · Zhichao Yan, Jiapu Wang, Jiaoyan Chen, XiaoLi Li 외

Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, including two general paradigms: Retrieval-The…

Knowledge Graphs Question Answering Retrieval

Legal RAG Bench: an end-to-end benchmark for legal RAG

2026-03-02 · Abdur-Rahman Butler, Umar Butler arxiv

We introduce Legal RAG Bench, a benchmark and evaluation methodology for assessing the end-to-end performance of legal RAG systems. As a benchmark, Legal RAG Bench consists of 4,876 passages from the Victorian Criminal C…

Information Retrieval