Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
Modern language models frequently include retrieval components to improve their outputs, giving rise to a growing number of retrieval-augmented generation (RAG) systems. Yet, most existing work in RAG has underemphasized fair ranking techniques and neglected the diverse interests of all stakeholders. In this paper, we present the first comprehensive study of RAG systems that incorporate fairness-aware rankings, focusing on both ranking fairness and attribution fairness - ensuring equitable exposure of sources cited in the final text. We specifically examine item-side fairness, i.e., whether retrieved documents receive balanced exposure, and assess how this affects both the system's overall performance and the eventual distribution of cited sources. Across twelve RAG models and seven tasks, we find that fairness-aware retrieval frequently retains or even improves ranking effectiveness and generation quality, countering the widespread belief that fairness compromises system performance. Moreover, we show that fair retrieval leads to more balanced attribution in the final responses, ensuring that the cited sources are credited more equitably. Our results underscore the importance of item-side fairness throughout both retrieval and generation phases, offering key insights for building more responsible and equitable RAG systems and illustrating promising avenues for future exploration in fair ranking and source attribution.
Code (1)
Tasks
FairnessRAGRetrievalRetrieval-augmented GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fair ranking: a critical review, challenges, and future directions
Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fa…
FairnessRetrievalToward Automatic Group Membership Annotation for Group Fairness Evaluation
With the increasing research attention on fairness in information retrieval systems, more and more fairness-aware algorithms have been proposed to ensure fairness for a sustainable and healthy retrieval ecosystem. Howeve…
FairnessInformation RetrievalRetrievalThe Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant document from external knowledge sources. By referencing this external knowledge, RAG effectively reduces the generation o…
FairnessHallucinationRAGRetrieval+1Overview of the TREC 2022 Fair Ranking Track
The TREC Fair Ranking Track aims to provide a platform for participants to develop and evaluate novel retrieval algorithms that can provide a fair exposure to a mixture of demographics or attributes, such as ethnicity, t…
RetrievalOverview of the TREC 2021 Fair Ranking Track
The TREC Fair Ranking Track aims to provide a platform for participants to develop and evaluate novel retrieval algorithms that can provide a fair exposure to a mixture of demographics or attributes, such as ethnicity, t…
Retrieval