Curvature-aware Expected Free Energy as an Acquisition Function for Bayesian Optimization
We propose an Expected Free Energy-based acquisition function for Bayesian optimization to solve the joint learning and optimization problem, i.e., optimize and learn the underlying function simultaneously. We show that, under specific assumptions, Expected Free Energy reduces to Upper Confidence Bound, Lower Confidence Bound, and Expected Information Gain. We prove that Expected Free Energy has unbiased convergence guarantees for concave functions. Using the results from these derivations, we introduce a curvature-aware update law for Expected Free Energy and show its proof of concept using a system identification problem on a Van der Pol oscillator. Through rigorous simulation experiments, we show that our adaptive Expected Free Energy-based acquisition function outperforms state-of-the-art acquisition functions with the least final simple regret and error in learning the Gaussian process.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization
Active data acquisition is central to many learning and optimization tasks in deep neural networks, yet remains challenging because most approaches rely on predictive uncertainty estimates that are difficult to obtain re…
Text ClassificationActive LearningModel Merging on Loss Landscape: A Geometry Perspective
Model merging offers a promising avenue for knowledge integration and parallel development without retraining. Yet, existing methods either ignore the geometry of the loss landscape or rely on intractable full-space Hess…
Image ClassificationSmoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention
Conditional diffusion models have shown remarkable success in visual content generation, producing high-quality samples across various domains, largely due to classifier-free guidance (CFG). Recent attempts to extend gui…
Image GenerationThe impact of surface area, volume, curvature and Lennard-Jones potential to solvation modeling
This paper explores the impact of surface area, volume, curvature and Lennard-Jones potential on solvation free energy predictions. Rigidity surfaces are utilized to generate robust analytical expressions for maximum, mi…
Reframing the Expected Free Energy: Four Formulations and a Unification
Active inference is a leading theory of perception, learning and decision making, which can be applied to neuroscience, robotics, psychology, and machine learning. Active inference is based on the expected free energy, w…
Decision Making