Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How
Optimizing an objective function with uncertainty awareness is well-known to improve the accuracy and confidence of optimization solutions. Meanwhile, another relevant but very different question remains yet open: how to model and quantify the uncertainty of an optimization algorithm (a.k.a., optimizer) itself? To close such a gap, the prerequisite is to consider the optimizers as sampled from a distribution, rather than a few prefabricated and fixed update rules. We first take the novel angle to consider the algorithmic space of optimizers, and provide definitions for the optimizer prior and likelihood, that intrinsically determine the posterior and therefore uncertainty. We then leverage the recent advance of learning to optimize (L2O) for the space parameterization, with the end-to-end training pipeline built via variational inference, referred as uncertainty-aware L2O (UA-L2O). Our study represents the first effort to recognize and quantify the uncertainty of the optimization algorithm. The extensive numerical results show that, UA-L2O achieves superior optimization performances in two out of three test functions, the loss function in data privacy attack, and the energy function in docking four out of five protein pairs, with the most accurate posterior estimation. Besides, the optimized solutions close to the real solution are of a larger population and tighter confidence intervals, demonstrating a better calibration capability. All our codes will be publicly released upon acceptance.
Code (0)
등록된 구현이 없습니다.
Tasks
Uncertainty QuantificationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generative Parameter Sampler For Scalable Uncertainty Quantification
Uncertainty quantification has been a core of the statistical machine learning, but its computational bottleneck has been a serious challenge for both Bayesians and frequentists. We propose a model-based framework in qua…
General ClassificationUncertainty QuantificationBayesian Neural Hawkes Process for Event Uncertainty Prediction
Event data consisting of time of occurrence of the events arises in several real-world applications. Recent works have introduced neural network based point processes for modeling event-times, and were shown to provide s…
Decision MakingPoint ProcessesPredictionUncertainty QuantificationIncorporating uncertainty quantification into travel mode choice modeling: a Bayesian neural network (BNN) approach and an uncertainty-guided active survey framework
Existing deep learning approaches for travel mode choice modeling fail to inform modelers about their prediction uncertainty. Even when facing scenarios that are out of the distribution of training data, which implies hi…
Explainable artificial intelligencePredictionSurveyUncertainty QuantificationBayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling
Accurate predictions and uncertainty quantification (UQ) are essential for decision-making in risk-sensitive fields such as system safety modeling. Deep ensembles (DEs) are efficient and scalable methods for UQ in Deep N…
Bayesian OptimizationUncertainty QuantificationFully Bayesian VIB-DeepSSM
Statistical shape modeling (SSM) enables population-based quantitative analysis of anatomical shapes, informing clinical diagnosis. Deep learning approaches predict correspondence-based SSM directly from unsegmented 3D i…
AnatomyUncertainty QuantificationVariational Inference