Amortized Active Learning for Nonparametric Functions

2024-07-25 · Cen-You Li, Marc Toussaint, Barbara Rakitsch, Christoph Zimmer

Active learning (AL) is a sequential learning scheme aiming to select the most informative data. AL reduces data consumption and avoids the cost of labeling large amounts of data. However, AL trains the model and solves an acquisition optimization for each selection. It becomes expensive when the model training or acquisition optimization is challenging. In this paper, we focus on active nonparametric function learning, where the gold standard Gaussian process (GP) approaches suffer from cubic time complexity. We propose an amortized AL method, where new data are suggested by a neural network which is trained up-front without any real data (Figure 1). Our method avoids repeated model training and requires no acquisition optimization during the AL deployment. We (i) utilize GPs as function priors to construct an AL simulator, (ii) train an AL policy that can zero-shot generalize from simulation to real learning problems of nonparametric functions and (iii) achieve real-time data selection and comparable learning performances to time-consuming baseline methods.

📄 PDF Abstract BibTeX arXiv:2407.17992

Code (0)

등록된 구현이 없습니다.

Tasks

Active Learning

Methods 이 논문이 사용한 방법론

GPS Greedy Policy Search (GPS) is a simple algorithm that learns a policy for test-time data augmentation based on the predictive performance on a validation set. GPS starts with…

Focus 설명 없음

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies from Simulated Nonparametric Functions

2025-01-26 · Cen-You Li, Marc Toussaint, Barbara Rakitsch, Christoph Zimmer

Safe active learning (AL) is a sequential scheme for learning unknown systems while respecting safety constraints during data acquisition. Existing methods often rely on Gaussian processes (GPs) to model the task and saf…

Active Learning Decision Making Experimental Design Gaussian Processes

Amortized Variational Inference for Deep Gaussian Processes

2024-09-18 · Qiuxian Meng, Yongyou Zhang

Gaussian processes (GPs) are Bayesian nonparametric models for function approximation with principled predictive uncertainty estimates. Deep Gaussian processes (DGPs) are multilayer generalizations of GPs that can repres…

Gaussian Processes Variational Inference

Random Function Priors for Correlation Modeling

2019-05-09 · Aonan Zhang, John Paisley

The likelihood model of high dimensional data $X_n$ can often be expressed as $p(X_n|Z_n,\theta)$, where $\theta\mathrel{\mathop:}=(\theta_k)_{k\in[K]}$ is a collection of hidden features shared across objects, indexed b…

Variational Inference

S2: An Efficient Graph Based Active Learning Algorithm with Application to Nonparametric Classification

2015-06-29 · Gautam Dasarathy, Robert Nowak, Xiaojin Zhu

This paper investigates the problem of active learning for binary label prediction on a graph. We introduce a simple and label-efficient algorithm called S2 for this task. At each step, S2 selects the vertex to be labele…

Active Learning General Classification

Nuances in Margin Conditions Determine Gains in Active Learning

2021-10-16 · Samory Kpotufe, Gan Yuan, Yunfan Zhao

We consider nonparametric classification with smooth regression functions, where it is well known that notions of margin in $E[Y|X]$ determine fast or slow rates in both active and passive learning. Here we elucidate a s…