SoQal: Selective Oracle Questioning for Consistency Based Active Learning of Cardiac Signals
Clinical settings are often characterized by abundant unlabelled data and limited labelled data. This is typically driven by the high burden placed on oracles (e.g., physicians) to provide annotations. One way to mitigate this burden is via active learning (AL) which involves the (a) acquisition and (b) annotation of informative unlabelled instances. Whereas previous work addresses either one of these elements independently, we propose an AL framework that addresses both. For acquisition, we propose Bayesian Active Learning by Consistency (BALC), a sub-framework which perturbs both instances and network parameters and quantifies changes in the network output probability distribution. For annotation, we propose SoQal, a sub-framework that dynamically determines whether, for each acquired unlabelled instance, to request a label from an oracle or to pseudo-label it instead. We show that BALC can outperform start-of-the-art acquisition functions such as BALD, and SoQal outperforms baseline methods even in the presence of a noisy oracle.
Code (2)
Tasks
Active LearningPseudo LabelTime Series AnalysisSimilar Papers 제목 키워드 기반
SoQal: Selective Oracle Questioning in Active Learning
Large sets of unlabelled data within the healthcare domain remain underutilized. Active learning offers a way to exploit these datasets by iteratively requesting an oracle (e.g. medical professional) to label instances. …
Active LearningSoCal: Selective Oracle Questioning for Consistency-based Active Learning of Physiological Signals
The ubiquity and rate of collection of physiological signals produce large, unlabelled datasets. Active learning (AL) can exploit such datasets by incorporating human annotators (oracles) to improve generalization perfor…
Active LearningPseudo LabelCategory-Based Strategy-Driven Question Generator for Visual Dialogue
“GuessWhat?! is a task-oriented visual dialogue task which has two players a guesser and anoracle. Guesser aims to locate the object supposed by oracle by asking several Yes/No questions which are answered by oracle. How…
SentencePICon: A Multi-Turn Interrogation Framework for Evaluating Persona Agent Consistency
Large language model (LLM)-based persona agents are rapidly being adopted as scalable proxies for human participants across diverse domains. Yet there is no systematic method for verifying whether a persona agent's respo…
Selective Sampling and Imitation Learning via Online Regression
We consider the problem of Imitation Learning (IL) by actively querying noisy expert for feedback. While imitation learning has been empirically successful, much of prior work assumes access to noiseless expert feedback …
Imitation Learningregression