On the Metric Properties of IR Evaluation Measures Based on Ranking Axioms
The axiomatic analysis of IR evaluation metrics has contributed to a better understanding of their properties. Some works have modelled the effectiveness of retrieval measures with axioms that capture desirable properties on the set of ranked lists of documents. Recently, it has been shown that three of these axioms lead to some orderings. This work formally explores the metric properties of the set of rankings, endowed with these orderings. Based on lattice theory, the possible metrics and pseudo-metrics, defined on these structures, are determined. It is found that, when the relevant documents are prioritized, precision, recall, RBP and DCG are metrics on the set of rankings, however they are pseudo-metrics when the swapping of documents is considered.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSimilar Papers 제목 키워드 기반
On the Effect of Ranking Axioms on IR Evaluation Metrics
The study of IR evaluation metrics through axiomatic analysis enables a better understanding of their numerical properties. Some works have modelled the effectiveness of retrieval metrics with axioms that capture desirab…
RetrievalAn Axiomatic Approach to Regularizing Neural Ranking Models
Axiomatic information retrieval (IR) seeks a set of principle properties desirable in IR models. These properties when formally expressed provide guidance in the search for better relevance estimation functions. Neural r…
Information Retrievalparameter estimationRetrievalMeasuring Diversity: Axioms and Challenges
The concept of diversity is widely used in various applications: from image or molecule generation to recommender systems. Thus, being able to properly measure diversity is important. This paper addresses the problem of …
DiversityRecommendation SystemsFoundations of the Theory of Performance-Based Ranking
Ranking entities such as algorithms, devices, methods, or models based on their performances, while accounting for application-specific preferences, is a challenge. To address this challenge, we establish the foundations…
SpecificityDiversification quotients: Quantifying diversification via risk measures
We establish the first axiomatic theory for diversification indices using six intuitive axioms: non-negativity, location invariance, scale invariance, rationality, normalization, and continuity. The unique class of indic…
Portfolio Optimization