Continuation Path Learning for Homotopy Optimization
Homotopy optimization is a traditional method to deal with a complicated optimization problem by solving a sequence of easy-to-hard surrogate subproblems. However, this method can be very sensitive to the continuation schedule design and might lead to a suboptimal solution to the original problem. In addition, the intermediate solutions, often ignored by classic homotopy optimization, could be useful for many real-world applications. In this work, we propose a novel model-based approach to learn the whole continuation path for homotopy optimization, which contains infinite intermediate solutions for any surrogate subproblems. Rather than the classic unidirectional easy-to-hard optimization, our method can simultaneously optimize the original problem and all surrogate subproblems in a collaborative manner. The proposed model also supports real-time generation of any intermediate solution, which could be desirable for many applications. Experimental studies on different problems show that our proposed method can significantly improve the performance of homotopy optimization and provide extra helpful information to support better decision-making.
Code (1)
Tasks
Decision MakingSimilar Papers 제목 키워드 기반
Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization
We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients,…
Homotopy Analysis for Tensor PCA
Developing efficient and guaranteed nonconvex algorithms has been an important challenge in modern machine learning. Algorithms with good empirical performance such as stochastic gradient descent often lack theoretical g…
global-optimizationParameter Continuation Methods for the Optimization of Deep Neural Networks
There are many extant methods for approximating the solutions of non-convex optimization problems arising in deep neural networks, including stochastic gradient descent, RMSProp, AdaGrad, and ADAM. In this paper, we prop…
A fast algorithm for solving the lasso problem exactly without homotopy using differential inclusions
We prove in this work that the well-known lasso problem can be solved exactly without homotopy using novel differential inclusions techniques. Specifically, we show that a selection principle from the theory of different…
HUNet: Homotopy Unfolding Network for Image Compressive Sensing
Deep Unfolding Networks (DUNs) have risen to prominence due to their interpretability and superior performance for image Compressive Sensing (CS). However, existing DUNs still face significant issues, such as the ins…
Compressive SensingImage Reconstruction