Beyond Comparing Image Pairs: Setwise Active Learning for Relative Attributes
It is useful to automatically compare images based on their visual properties---to predict which image is brighter, more feminine, more blurry, etc. However, comparative models are inherently more costly to train than their classification counterparts. Manually labeling all pairwise comparisons is intractable, so which pairs should a human supervisor compare? We explore active learning strategies for training relative attribute ranking functions, with the goal of requesting human comparisons only where they are most informative. We introduce a novel criterion that requests a partial ordering for a set of examples that minimizes the total rank margin in attribute space, subject to a visual diversity constraint. The setwise criterion helps amortize effort by identifying mutually informative comparisons, and the diversity requirement safeguards against requests a human viewer will find ambiguous. We develop an efficient strategy to search for sets that meet this criterion. On three challenging datasets and experiments with "live" online annotators, the proposed method outperforms both traditional passive learning as well as existing active rank learning methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningAttributeDiversitySimilar Papers 제목 키워드 기반
Beyond Reproducibility: Advancing Zero-shot LLM Reranking Efficiency with Setwise Insertion
This study presents a comprehensive reproducibility and extension analysis of the Setwise prompting methodology for zero-shot ranking with Large Language Models (LLMs), as proposed by Zhuang et al. We evaluate its effect…
Computational EfficiencyDocument RankingRerankingConstructive Preference Elicitation by Setwise Max-margin Learning
In this paper we propose an approach to preference elicitation that is suitable to large configuration spaces beyond the reach of existing state-of-the-art approaches. Our setwise max-margin method can be viewed as a gen…
Beyond Pairwise Comparisons in Social Choice: A Setwise Kemeny Aggregation Problem
In this paper, we advocate the use of setwise contests for aggregating a set of input rankings into an output ranking. We propose a generalization of the Kemeny rule where one minimizes the number of k-wise disagreements…
Optimal majority rules and quantitative Condorcet properties of setwise Kemeny voting schemes
The important Kemeny problem, which consists of computing median consensus rankings of an election with respect to the Kemeny voting rule, admits important applications in biology and computational social choice and was …
Quasi-stability notions in two-sided matching models
This paper presents weakened notions of corewise stability and setwise stability for matching markets where agents have substitutable choice functions. We introduce the concepts of worker-quasi-core, firm-quasi-core, and…