Interpreting Neural Ranking Models using Grad-CAM
Recently, applying deep neural networks in IR has become an important and timely topic. For instance, Neural Ranking Models(NRMs) have shown promising performance compared to the traditional ranking models. However, explaining the ranking results has become even more difficult with NRM due to the complex structure of neural networks. On the other hand, a great deal of research is under progress on Interpretable Machine Learning(IML), including Grad-CAM. Grad-CAM is an attribution method and it can visualize the input regions that contribute to the network's output. In this paper, we adopt Grad-CAM for interpreting the ranking results of NRM. By adopting Grad-CAM, we analyze how each query-document term pair contributes to the matching score for a given pair of query and document. The visualization results provide insights on why a certain document is relevant to the given query. Also, the results show that neural ranking model captures the subtle notion of relevance. Our interpretation method and visualization results can be used for snippet generation and user-query intent analysis.
Code (0)
등록된 구현이 없습니다.
Tasks
Interpretable Machine LearningSimilar Papers 제목 키워드 기반
Interpreting Questions with a Log-Linear Ranking Model in a Virtual Patient Dialogue System
Finite-time Analysis for the Knowledge-Gradient Policy
We consider sequential decision problems in which we adaptively choose one of finitely many alternatives and observe a stochastic reward. We offer a new perspective of interpreting Bayesian ranking and selection problems…
Explaining Ranking Models using Multiple Explainers
Current approaches to interpreting complex ranking models are based on local approximations of the ranking model using a simple ranker in the locality of the query. Since rankings have multiple relevance factors and are …
How Relevance Emerges: Interpreting LoRA Fine-Tuning in Reranking LLMs
We conduct a behavioral exploration of LoRA fine-tuned LLMs for Passage Reranking to understand how relevance signals are learned and deployed by Large Language Models. By fine-tuning Mistral-7B, LLaMA3.1-8B, and Pythia-…
Information RetrievalPassage RerankingRerankingRetrievalLearning Preference from Observed Rankings
Estimating consumer preferences is central to many problems in economics and marketing. This paper develops a flexible framework for learning individual preferences from partial ranking information by interpreting observ…