paper-with-me

Papers

Active Ranking using Pairwise Comparisons

2011-09-16 · NeurIPS 2011 12 · Kevin G. Jamieson, Robert D. Nowak

This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In general, the ranking of $n$ objects can be identified by standard sorting methods using $n log_2 n$ pairwise comparisons. We are interested in natural situations in which relationships among the objects may allow for ranking using far fewer pairwise comparisons. Specifically, we assume that the objects can be embedded into a $d$-dimensional Euclidean space and that the rankings reflect their relative distances from a common reference point in $R^d$. We show that under this assumption the number of possible rankings grows like $n^{2d}$ and demonstrate an algorithm that can identify a randomly selected ranking using just slightly more than $d log n$ adaptively selected pairwise comparisons, on average. If instead the comparisons are chosen at random, then almost all pairwise comparisons must be made in order to identify any ranking. In addition, we propose a robust, error-tolerant algorithm that only requires that the pairwise comparisons are probably correct. Experimental studies with synthetic and real datasets support the conclusions of our theoretical analysis.

📄 PDF Abstract BibTeX arXiv:1109.3701

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Decreasing Annotation Burden of Pairwise Comparisons with Human-in-the-Loop Sorting: Application in Medical Image Artifact Rating

2022-02-10 · Ikbeom Jang, Garrison Danley, Ken Chang, Jayashree Kalpathy-Cramer

Ranking by pairwise comparisons has shown improved reliability over ordinal classification. However, as the annotations of pairwise comparisons scale quadratically, this becomes less practical when the dataset is large. …

Ordinal Classification

Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Help

2016-06-28 · Reinhard Heckel, Nihar B. Shah, Kannan Ramchandran, Martin J. Wainwright

We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partit…

Open-Ended Question Answering

Approximate Ranking from Pairwise Comparisons

2018-01-04 · Reinhard Heckel, Max Simchowitz, Kannan Ramchandran, Martin J. Wainwright

A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes ide…

Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons

2021-10-08 · Yue Wu, Tao Jin, Hao Lou, Pan Xu 외

In heterogeneous rank aggregation problems, users often exhibit various accuracy levels when comparing pairs of items. Thus a uniform querying strategy over users may not be optimal. To address this issue, we propose an …

Active Learners as Efficient PRP Rerankers

2026-05-14 · Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero, Santiago Barron 외 arxiv

Pairwise Ranking Prompting (PRP) elicits pairwise preference judgments from an LLM, which are then aggregated into a ranking, usually via classical sorting algorithms. However, judgments are noisy, order-sensitive, and s…

Active Learning