Probabilistic computation and uncertainty quantification with emerging covariance
Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems.
Code (1)
Tasks
Uncertainty QuantificationSimilar Papers 제목 키워드 기반
Multivariate Probabilistic Time Series Forecasting with Correlated Errors
Accurately modeling the correlation structure of errors is critical for reliable uncertainty quantification in probabilistic time series forecasting. While recent deep learning models for multivariate time series have de…
Computational EfficiencyProbabilistic Time Series ForecastingTime SeriesTime Series Forecasting+1Better Batch for Deep Probabilistic Time Series Forecasting
Deep probabilistic time series forecasting has gained attention for its ability to provide nonlinear approximation and valuable uncertainty quantification for decision-making. However, existing models often oversimplify …
Decision MakingProbabilistic Time Series ForecastingTime SeriesTime Series Forecasting+1Uncertainty Quantification in Working Memory via Moment Neural Networks
Humans possess a finely tuned sense of uncertainty that helps anticipate potential errors, vital for adaptive behavior and survival. However, the underlying neural mechanisms remain unclear. This study applies moment neu…
Uncertainty QuantificationUncertainty Quantification for Quantum Computing
This review is designed to introduce mathematicians and computational scientists to quantum computing (QC) through the lens of uncertainty quantification (UQ) by presenting a mathematically rigorous and accessible narrat…
Bayesian InferenceVariational bagging: a robust approach for Bayesian uncertainty quantification
Variational Bayes methods are popular due to their computational efficiency and adaptability to diverse applications. In specifying the variational family, mean-field classes are commonly used, which enables efficient al…
Computational Efficiency