Unsupervised Evaluation and Weighted Aggregation of Ranked Predictions
Learning algorithms that aggregate predictions from an ensemble of diverse base classifiers consistently outperform individual methods. Many of these strategies have been developed in a supervised setting, where the accuracy of each base classifier can be empirically measured and this information is incorporated in the training process. However, the reliance on labeled data precludes the application of ensemble methods to many real world problems where labeled data has not been curated. To this end we developed a new theoretical framework for binary classification, the Strategy for Unsupervised Multiple Method Aggregation (SUMMA), to estimate the performances of base classifiers and an optimal strategy for ensemble learning from unlabeled data.
Code (2)
Tasks
Binary ClassificationEnsemble LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
Can we aggregate human intelligence? an approach for human centric aggregation using ordered weighted averaging operators
The primary objective of this paper is to present an approach for recommender systems that can assimilate ranking to the voters or rankers so that recommendation can be made by giving priority to experts suggestion over …
Decision MakingRecommendation SystemsMeasuring Bias in a Ranked List using Term-based Representations
In most recent studies, gender bias in document ranking is evaluated with the NFaiRR metric, which measures bias in a ranked list based on an aggregation over the unbiasedness scores of each ranked document. This perspec…
Document RankingFairnessPassage RankingGradual Argumentation Evaluation for Stance Aggregation in Automated Fake News Detection
Stance detection plays a pivot role in fake news detection. The task involves determining the point of view or stance {--} for or against {--} a text takes towards a claim. One very important stage in employing stance de…
Fake News DetectionStance DetectionMultivariate Spearman's rho for aggregating ranks using copulas
We study the problem of rank aggregation: given a set of ranked lists, we want to form a consensus ranking. Furthermore, we consider the case of extreme lists: i.e., only the rank of the best or worst elements are known.…
Learning-To-RankTheme-weighted Ranking of Keywords from Text Documents using Phrase Embeddings
Keyword extraction is a fundamental task in natural language processing that facilitates mapping of documents to a concise set of representative single and multi-word phrases. Keywords from text documents are primarily e…
Keyword Extraction