Accelerating Optimization via Adaptive Prediction
We present a powerful general framework for designing data-dependent optimization algorithms, building upon and unifying recent techniques in adaptive regularization, optimistic gradient predictions, and problem-dependent randomization. We first present a series of new regret guarantees that hold at any time and under very minimal assumptions, and then show how different relaxations recover existing algorithms, both basic as well as more recent sophisticated ones. Finally, we show how combining adaptivity, optimism, and problem-dependent randomization can guide the design of algorithms that benefit from more favorable guarantees than recent state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionSimilar Papers 제목 키워드 기반
An accelerate Prediction Strategy for Dynamic Multi-Objective Optimization
This paper addresses the challenge of dynamic multi-objective optimization problems (DMOPs) by introducing novel approaches for accelerating prediction strategies within the evolutionary algorithm framework. Since the ob…
PredictionAcceleration in Policy Optimization
We work towards a unifying paradigm for accelerating policy optimization methods in reinforcement learning (RL) by integrating foresight in the policy improvement step via optimistic and adaptive updates. Leveraging the …
Meta-LearningPolicy Gradient MethodsReinforcement Learning (RL)P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling
Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text, imposing a significant latency bottleneck, particularly for token-dense documents. While M…
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
Heavy ball momentum is crucial in accelerating (stochastic) gradient-based optimization algorithms for machine learning. Existing heavy ball momentum is usually weighted by a uniform hyperparameter, which relies on exces…
BIG-bench Machine Learningimage-classificationImage ClassificationLanguage Modeling+3Accelerating genetic optimization of nonlinear model predictive control by learning optimal search space size
Genetic algorithm (GA) is typically used to solve nonlinear model predictive control's optimization problem. However, the size of the search space in which the GA searches for the optimal control inputs is crucial for it…
Evolutionary AlgorithmsModel Predictive Control