Conditionally Gaussian PAC-Bayes
Recent studies have empirically investigated different methods to train stochastic neural networks on a classification task by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that this approach outperforms currently available PAC-Bayesian training methods.
Code (1)
Similar Papers 제목 키워드 기반
High-Dimensional Conditionally Gaussian State Space Models with Missing Data
We develop an efficient sampling approach for handling complex missing data patterns and a large number of missing observations in conditionally Gaussian state space models. Two important examples are dynamic factor mode…
State Space ModelsVocal Bursts Intensity PredictionA Bayesian Perspective on the Maximum Score Problem
This paper presents a Bayesian inference framework for a linear index threshold-crossing binary choice model that satisfies a median independence restriction. The key idea is that the model is observationally equivalent …
Bayesian InferenceLeveraging joint sparsity in hierarchical Bayesian learning
We present a hierarchical Bayesian learning approach to infer jointly sparse parameter vectors from multiple measurement vectors. Our model uses separate conditionally Gaussian priors for each parameter vector and common…
Bayesian InferenceEntropic bounds for conditionally Gaussian vectors and applications to neural networks
Using entropic inequalities from information theory, we provide new bounds on the total variation and 2-Wasserstein distances between a conditionally Gaussian law and a Gaussian law with invertible covariance matrix. We …
Modeling Neural Activity with Conditionally Linear Dynamical Systems
Neural population activity exhibits complex, nonlinear dynamics, varying in time, over trials, and across experimental conditions. Here, we develop Conditionally Linear Dynamical System (CLDS) models as a general-purpose…
Bayesian Inference