Towards Axiomatic Explanations for Neural Ranking Models
Recently, neural networks have been successfully employed to improve upon state-of-the-art performance in ad-hoc retrieval tasks via machine-learned ranking functions. While neural retrieval models grow in complexity and impact, little is understood about their correspondence with well-studied IR principles. Recent work on interpretability in machine learning has provided tools and techniques to understand neural models in general, yet there has been little progress towards explaining ranking models. We investigate whether one can explain the behavior of neural ranking models in terms of their congruence with well understood principles of document ranking by using established theories from axiomatic IR. Axiomatic analysis of information retrieval models has formalized a set of constraints on ranking decisions that reasonable retrieval models should fulfill. We operationalize this axiomatic thinking to reproduce rankings based on combinations of elementary constraints. This allows us to investigate to what extent the ranking decisions of neural rankers can be explained in terms of retrieval axioms, and which axioms apply in which situations. Our experimental study considers a comprehensive set of axioms over several representative neural rankers. While the existing axioms can already explain the particularly confident ranking decisions rather well, future work should extend the axiom set to also cover the other still "unexplainable" neural IR rank decisions.
Code (0)
등록된 구현이 없습니다.
Tasks
Document RankingInformation RetrievalRetrievalSimilar Papers 제목 키워드 기반
RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank
Numerous works propose post-hoc, model-agnostic explanations for learning to rank, focusing on ordering entities by their relevance to a query through feature attribution methods. However, these attributions often weakly…
Computational EfficiencyLearning-To-RankAn Axiomatic Approach to Regularizing Neural Ranking Models
Axiomatic information retrieval (IR) seeks a set of principle properties desirable in IR models. These properties when formally expressed provide guidance in the search for better relevance estimation functions. Neural r…
Information Retrievalparameter estimationRetrievalModel Explanations via the Axiomatic Causal Lens
Explaining the decisions of black-box models is a central theme in the study of trustworthy ML. Numerous measures have been proposed in the literature; however, none of them take an axiomatic approach to causal explainab…
Feature ImportancemodelAxiomatic Aggregations of Abductive Explanations
The recent criticisms of the robustness of post hoc model approximation explanation methods (like LIME and SHAP) have led to the rise of model-precise abductive explanations. For each data point, abductive explanations p…
Feature ImportancevalidA modified axiomatic foundation of the analytic hierarchy process
This paper reports a modified axiomatic foundation of the analytic hierarchy process (AHP), where the reciprocal property of paired comparisons is broken. The novel concept of reciprocal symmetry breaking is proposed to …