Challenges in High-dimensional Reinforcement Learning with Evolution Strategies

2018-06-04 · Nils Müller, Tobias Glasmachers

Evolution Strategies (ESs) have recently become popular for training deep neural networks, in particular on reinforcement learning tasks, a special form of controller design. Compared to classic problems in continuous direct search, deep networks pose extremely high-dimensional optimization problems, with many thousands or even millions of variables. In addition, many control problems give rise to a stochastic fitness function. Considering the relevance of the application, we study the suitability of evolution strategies for high-dimensional, stochastic problems. Our results give insights into which algorithmic mechanisms of modern ES are of value for the class of problems at hand, and they reveal principled limitations of the approach. They are in line with our theoretical understanding of ESs. We show that combining ESs that offer reduced internal algorithm cost with uncertainty handling techniques yields promising methods for this class of problems.

📄 PDF Abstract BibTeX arXiv:1806.01224

Code (1)

NiMlr/High-Dim-ES-RL 공식 구현 tf

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Intensity Prediction

CoNES: Convex Natural Evolutionary Strategies

2020-07-16 · Sushant Veer, Anirudha Majumdar

We present a novel algorithm -- convex natural evolutionary strategies (CoNES) -- for optimizing high-dimensional blackbox functions by leveraging tools from convex optimization and information geometry. CoNES is formula…

Benchmarking MuJoCo reinforcement-learning Reinforcement Learning (RL)

Solving Deep Reinforcement Learning Tasks with Evolution Strategies and Linear Policy Networks

2024-02-10 · Annie Wong, Jacob de Nobel, Thomas Bäck, Aske Plaat 외

Although deep reinforcement learning methods can learn effective policies for challenging problems such as Atari games and robotics tasks, algorithms are complex, and training times are often long. This study investigate…

Atari Games Deep Reinforcement Learning Q-Learning reinforcement-learning+1

Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning

2025-09-29 · Xin Qiu, Yulu Gan, Conor F. Hayes, Qiyao Liang 외 arxiv

Fine-tuning large language models (LLMs) for downstream tasks is an essential stage of modern AI deployment. Reinforcement learning (RL) has emerged as the dominant fine-tuning paradigm, underpinning many state-of-the-ar…

Dimensionality Reduction Reinforcement Learning

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

2021-09-28 · Amjad Yousef Majid, Serge Saaybi, Tomas van Rietbergen, Vincent Francois-Lavet 외

Deep Reinforcement Learning (DRL) and Evolution Strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths an…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+3

Meta Reinforcement Learning with Distribution of Exploration Parameters Learned by Evolution Strategies

2018-12-29 · Yiming Shen, Kehan Yang, Yufeng Yuan, Simon Cheng Liu

In this paper, we propose a novel meta-learning method in a reinforcement learning setting, based on evolution strategies (ES), exploration in parameter space and deterministic policy gradients. ES methods are easy to pa…

Meta-Learning Meta Reinforcement Learning MuJoCo reinforcement-learning+2