Test Set Selection using Active Information Acquisition for Predictive Models
In this paper, we consider active information acquisition when the prediction model is meant to be applied on a targeted subset of the population. The goal is to label a pre-specified fraction of customers in the target or test set by iteratively querying for information from the non-target or training set. The number of queries is limited by an overall budget. Arising in the context of two rather disparate applications- banking and medical diagnosis, we pose the active information acquisition problem as a constrained optimization problem. We propose two greedy iterative algorithms for solving the above problem. We conduct experiments with synthetic data and compare results of our proposed algorithms with few other baseline approaches. The experimental results show that our proposed approaches perform better than the baseline schemes.
Code (0)
등록된 구현이 없습니다.
Tasks
Medical DiagnosisSimilar Papers 제목 키워드 기반
Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variabl…
Active LearningExplaining Predictive Uncertainty with Information Theoretic Shapley Values
Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast, explaining the $\textit{uncertainty}$ of…
Active LearningDecision Making Under UncertaintyDepth Aleatoric Uncertainty EstimationExplainable Artificial Intelligence (XAI)+1Active Learning: Problem Settings and Recent Developments
In supervised learning, acquiring labeled training data for a predictive model can be very costly, but acquiring a large amount of unlabeled data is often quite easy. Active learning is a method of obtaining predictive m…
Active LearningRelaxed Efficient Acquisition of Context and Temporal Features
In many biomedical applications, measurements are not freely available at inference time: each laboratory test, imaging modality, or assessment incurs financial cost, time burden, or patient risk. Longitudinal active fea…
AFABench: A Generic Framework for Benchmarking Active Feature Acquisition
In many real-world scenarios, acquiring all features of a data instance can be expensive or impractical due to monetary cost, latency, or privacy concerns. Active Feature Acquisition (AFA) addresses this challenge by dyn…
Reinforcement Learning