Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control

2021-08-20 · Dimitri Bertsekas

In this paper we aim to provide analysis and insights (often based on visualization), which explain the beneficial effects of on-line decision making on top of off-line training. In particular, through a unifying abstract mathematical framework, we show that the principal AlphaZero/TD-Gammon ideas of approximation in value space and rollout apply very broadly to deterministic and stochastic optimal control problems, involving both discrete and continuous search spaces. Moreover, these ideas can be effectively integrated with other important methodologies such as model predictive control, adaptive control, decentralized control, discrete and Bayesian optimization, neural network-based value and policy approximations, and heuristic algorithms for discrete optimization.

📄 PDF Abstract BibTeX arXiv:2108.10315

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian OptimizationDecision MakingModel Predictive Control

Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming

2024-06-02 · Dimitri P. Bertsekas

In this paper we describe a new conceptual framework that connects approximate Dynamic Programming (DP), Model Predictive Control (MPC), and Reinforcement Learning (RL). This framework centers around two algorithms, whic…

Model Predictive Control Reinforcement Learning (RL)

Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning

2021-05-13 · Hui Wang, Mike Preuss, Aske Plaat

AlphaZero has achieved impressive performance in deep reinforcement learning by utilizing an architecture that combines search and training of a neural network in self-play. Many researchers are looking for ways to repro…

Board Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Targeted Search Control in AlphaZero for Effective Policy Improvement

2023-02-23 · Alexandre Trudeau, Michael Bowling

AlphaZero is a self-play reinforcement learning algorithm that achieves superhuman play in chess, shogi, and Go via policy iteration. To be an effective policy improvement operator, AlphaZero's search requires accurate v…

AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision

2026-07-09 · Brent Kong, Tejas Ram, Tony Yue Yu arxiv

AlphaZero has demonstrated that a neural-guided Monte Carlo Tree Search can achieve superhuman performance, but strong play does not necessarily imply perfect play. We study this gap in two oracle-evaluable domains with …

Lessons Learned from Data-Driven Building Control Experiments: Contrasting Gaussian Process-based MPC, Bilevel DeePC, and Deep Reinforcement Learning

2022-05-31 · Loris Di Natale, Yingzhao Lian, Emilio T. Maddalena, Jicheng Shi 외

This manuscript offers the perspective of experimentalists on a number of modern data-driven techniques: model predictive control relying on Gaussian processes, adaptive data-driven control based on behavioral theory, an…

Deep Reinforcement Learning Gaussian Processes Model Predictive Control reinforcement-learning+1