A closed loop gradient descent algorithm applied to Rosenbrock's function
We introduce a novel adaptive damping technique for an inertial gradient system which finds application as a gradient descent algorithm for unconstrained optimisation. In an example using the non-convex Rosenbrock's function, we show an improvement on existing momentum-based gradient optimisation methods. Also using Lyapunov stability analysis, we demonstrate the performance of the continuous-time version of the algorithm. Using numerical simulations, we consider the performance of its discrete-time counterpart obtained by using the symplectic Euler method of discretisation.
Code (1)
Similar Papers 제목 키워드 기반
Natural Gradient Descent for Control
This paper bridges optimization and control, and presents a novel closed-loop control framework based on natural gradient descent, offering a trajectory-oriented alternative to traditional cost-function tuning. By levera…
Finite-Time Gradient Descent-Based Adaptive Neural Network Finite-Time Control Design for Attitude Tracking of a 3-DOF Helicopter
This paper investigates a novel finite-time gradient descent-based adaptive neural network finite-time control strategy for the attitude tracking of a 3-DOF lab helicopter platform subject to composite disturbances. Firs…
Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control
We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dyna…
Whiplash Gradient Descent Dynamics
In this paper, we propose the Whiplash Inertial Gradient dynamics, a closed-loop optimization method that utilises gradient information, to find the minima of a cost function in finite-dimensional settings. We introduce …
A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
This paper studies the problem of stochastic bilevel optimization where the upper-level function is nonconvex with potentially unbounded smoothness and the lower-level function is strongly convex. This problem is motivat…
Bilevel OptimizationMeta-LearningStochastic Optimizationtext-classification+1