홈 › Papers

Effective Reinforcement Learning through Evolutionary Surrogate-Assisted Prescription

2020-02-13 · Olivier Francon, Santiago Gonzalez, Babak Hodjat, Elliot Meyerson, Risto Miikkulainen, Xin Qiu, Hormoz Shahrzad

There is now significant historical data available on decision making in organizations, consisting of the decision problem, what decisions were made, and how desirable the outcomes were. Using this data, it is possible to learn a surrogate model, and with that model, evolve a decision strategy that optimizes the outcomes. This paper introduces a general such approach, called Evolutionary Surrogate-Assisted Prescription, or ESP. The surrogate is, for example, a random forest or a neural network trained with gradient descent, and the strategy is a neural network that is evolved to maximize the predictions of the surrogate model. ESP is further extended in this paper to sequential decision-making tasks, which makes it possible to evaluate the framework in reinforcement learning (RL) benchmarks. Because the majority of evaluations are done on the surrogate, ESP is more sample efficient, has lower variance, and lower regret than standard RL approaches. Surprisingly, its solutions are also better because both the surrogate and the strategy network regularize the decision-making behavior. ESP thus forms a promising foundation to decision optimization in real-world problems.

📄 PDF Abstract BibTeX arXiv:2002.05368

Code (1)

lucylow/Covid_Control

Tasks

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making

Methods 이 논문이 사용한 방법론

Hierarchical Feature Fusion Hierarchical Feature Fusion (HFF) is a feature fusion method employed in ESP and EESP image…

Dilated Convolution 설명 없음

Pointwise Convolution Pointwise Convolution is a type of convolution that uses a 1x1 kernel: a kernel that iterates through every single point. This…

ESP 설명 없음

Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

2023-01-31 · Lan Tang, Xiaxi Li, Jinyuan Zhang, Guiying Li 외

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditiona…

Atari Games Evolutionary Algorithms reinforcement-learning Reinforcement Learning+1

Exploring the effectiveness of surrogate-assisted evolutionary algorithms on the batch processing problem

2022-10-31 · Mohamed Z. Variawa, Terence L. Van Zyl, Matthew Woolway

Real-world optimisation problems typically have objective functions which cannot be expressed analytically. These optimisation problems are evaluated through expensive physical experiments or simulations. Cheap approxima…

Evolutionary Algorithms

Surrogate-Assisted Evolutionary Reinforcement Learning Based on Autoencoder and Hyperbolic Neural Network

2025-05-26 · Bingdong Li, Mei Jiang, Peng Yang, Wenjing Hong 외

Evolutionary Reinforcement Learning (ERL), training the Reinforcement Learning (RL) policies with Evolutionary Algorithms (EAs), have demonstrated enhanced exploration capabilities and greater robustness than using tradi…

Evolutionary Algorithms MuJoCo MuJoCo Games Reinforcement Learning (RL)

Relation Reasoning with LLMs in Expensive Optimization

2026-04-30 · Ye Lu, Bingdong Li, Aimin Zhou, Hao Hao arxiv

Expensive optimization problems (EOPs) are black-box tasks with costly objective evaluations and no gradient access, making the evaluation budget the key bottleneck. Surrogate-assisted evolutionary algorithms (SAEAs) red…

A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning

2022-01-01 · Yuxing Wang, Tiantian Zhang, Yongzhe Chang, Bin Liang 외

The integration of Reinforcement Learning (RL) and Evolutionary Algorithms (EAs) aims at simultaneously exploiting the sample efficiency as well as the diversity and robustness of the two paradigms. Recently, hybrid lear…

continuous-control Continuous Control Diversity Evolutionary Algorithms+4