Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterization
Brittle optimization has been observed to adversely impact model likelihoods for regression and VAEs when simultaneously fitting neural network mappings from a (random) variable onto the mean and variance of a dependent Gaussian variable. Previous works have bolstered optimization and improved likelihoods, but fail other basic posterior predictive checks (PPCs). Under the PPC framework, we propose critiques to test predictive mean and variance calibration and the predictive distribution's ability to generate sensible data. We find that our attractively simple solution, to treat heteroscedastic variance variationally, sufficiently regularizes variance to pass these PPCs. We consider a diverse gamut of existing and novel priors and find our methods preserve or outperform existing model likelihoods while significantly improving parameter calibration and sample quality for regression and VAEs.
Code (2)
Tasks
regressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties
Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric uncertainties are learned as outputs of the BNN in addition to the p…
Variational InferenceFaithful Heteroscedastic Regression with Neural Networks
Heteroscedastic regression models a Gaussian variable's mean and variance as a function of covariates. Parametric methods that employ neural networks for these parameter maps can capture complex relationships in the data…
regressionVariational Heteroscedastic Volatility Model
We propose Variational Heteroscedastic Volatility Model (VHVM) -- an end-to-end neural network architecture capable of modelling heteroscedastic behaviour in multivariate financial time series. VHVM leverages recent adva…
modelRepresentation LearningTime SeriesTime Series AnalysisGrid-less Variational Direction of Arrival Estimation in Heteroscedastic Noise Environment
Horizontal line arrays are often employed in underwater environments to estimate the direction of arrival (DOA) of a weak signal. Conventional beamforming (CB) is robust but has wide beamwidths and high-level sidelobes. …
Direction of Arrival EstimationAccurate and Reliable Forecasting using Stochastic Differential Equations
It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. Although a lot of efforts have been made, such as heteroscedastic neural networks…
DiagnosticPrediction IntervalsUncertainty Quantification