Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training
A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multi-level feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behaviour in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Multilevel Reinforcement Learning Framework for PDE-based Control
Reinforcement learning (RL) is a promising method to solve control problems. However, model-free RL algorithms are sample inefficient and require thousands if not millions of samples to learn optimal control policies. A …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Multilevel Inverter Real-Time Simulation and Optimization Through Hybrid GA/PSO Algorithm
This paper presents a new real-time intelligent optimization algorithm to minimize the voltage harmonics of a multilevel inverter. Hybrid Genetic algorithm /Particle swarm optimization algorithm is employed in a real-tim…
A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression
Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A w…
regressionA multilevel thresholding algorithm using Electromagnetism Optimization
Segmentation is one of the most important tasks in image processing. It consist in classify the pixels into two or more groups depending on their intensity levels and a threshold value. The quality of the segmentation de…
Image SegmentationSegmentationSemantic SegmentationRecursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization
Two OFFO (Objective-Function Free Optimization) noise tolerant algorithms are presented that handle bound constraints, inexact gradients and use second-order information when available.The first is a multi-level method e…
Computational Efficiency