Mitigating Sampling Bias and Improving Robustness in Active Learning

2021-09-13 · Ranganath Krishnan, Alok Sinha, Nilesh Ahuja, Mahesh Subedar, Omesh Tickoo, Ravi Iyer

This paper presents simple and efficient methods to mitigate sampling bias in active learning while achieving state-of-the-art accuracy and model robustness. We introduce supervised contrastive active learning by leveraging the contrastive loss for active learning under a supervised setting. We propose an unbiased query strategy that selects informative data samples of diverse feature representations with our methods: supervised contrastive active learning (SCAL) and deep feature modeling (DFM). We empirically demonstrate our proposed methods reduce sampling bias, achieve state-of-the-art accuracy and model calibration in an active learning setup with the query computation 26x faster than Bayesian active learning by disagreement and 11x faster than CoreSet. The proposed SCAL method outperforms by a big margin in robustness to dataset shift and out-of-distribution.

📄 PDF Abstract BibTeX arXiv:2109.06321

Code (0)

등록된 구현이 없습니다.

Tasks

Active Learning

D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias

2023-05-26 · Sabit Hassan, Malihe Alikhani

Despite recent advancements, NLP models continue to be vulnerable to bias. This bias often originates from the uneven distribution of real-world data and can propagate through the annotation process. Escalated integratio…

Active Learning Clustering text-classification Text Classification

Mitigating sampling bias in risk-based active learning via an EM algorithm

2022-06-25 · Aidan J. Hughes, Lawrence A. Bull, Paul Gardner, Nikolaos Dervilis 외

Risk-based active learning is an approach to developing statistical classifiers for online decision-support. In this approach, data-label querying is guided according to the expected value of perfect information for inci…

Active Learning Decision Making

On robust risk-based active-learning algorithms for enhanced decision support

2022-01-07 · Aidan J. Hughes, Lawrence A. Bull, Paul Gardner, Nikolaos Dervilis 외

Classification models are a fundamental component of physical-asset management technologies such as structural health monitoring (SHM) systems and digital twins. Previous work introduced risk-based active learning, an on…

Active Learning Asset Management Decision Making Management+1

Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias

2022-05-27 · Sabina J. Sloman, Daniel M. Oppenheimer, Stephen B. Broomell, Cosma Rohilla Shalizi

Bayesian adaptive experimental design is a form of active learning, which chooses samples to maximize the information they give about uncertain parameters. Prior work has shown that other forms of active learning can suf…

Active Learning Experimental Design

Robustness analytics to data heterogeneity in edge computing

2020-02-12 · Jia Qian, Lars Kai Hansen, Xenofon Fafoutis, Prayag Tiwari 외

Federated Learning is a framework that jointly trains a model \textit{with} complete knowledge on a remotely placed centralized server, but \textit{without} the requirement of accessing the data stored in distributed mac…

Edge-computing Federated Learning