Learning to Do or Learning While Doing: Reinforcement Learning and Bayesian Optimisation for Online Continuous Tuning
Online tuning of real-world plants is a complex optimisation problem that continues to require manual intervention by experienced human operators. Autonomous tuning is a rapidly expanding field of research, where learning-based methods, such as Reinforcement Learning-trained Optimisation (RLO) and Bayesian optimisation (BO), hold great promise for achieving outstanding plant performance and reducing tuning times. Which algorithm to choose in different scenarios, however, remains an open question. Here we present a comparative study using a routine task in a real particle accelerator as an example, showing that RLO generally outperforms BO, but is not always the best choice. Based on the study's results, we provide a clear set of criteria to guide the choice of algorithm for a given tuning task. These can ease the adoption of learning-based autonomous tuning solutions to the operation of complex real-world plants, ultimately improving the availability and pushing the limits of operability of these facilities, thereby enabling scientific and engineering advancements.
Code (1)
Tasks
Bayesian OptimisationOpen-Ended Question AnsweringSimilar Papers 제목 키워드 기반
Ordinal Bayesian Optimisation
Bayesian optimisation is a powerful tool to solve expensive black-box problems, but fails when the stationary assumption made on the objective function is strongly violated, which is the case in particular for ill-condit…
Bayesian OptimisationThompson SamplingLarge Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language
Autonomous tuning of particle accelerators is an active and challenging field of research with the goal of enabling novel accelerator technologies cutting-edge high-impact applications, such as physics discovery, cancer …
Bayesian OptimisationSparse Adversarial Video Attacks with Spatial Transformations
In recent years, a significant amount of research efforts concentrated on adversarial attacks on images, while adversarial video attacks have seldom been explored. We propose an adversarial attack strategy on videos, cal…
Adversarial AttackBayesian OptimisationSSIMAlternating Optimisation and Quadrature for Robust Control
Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to impro…
Bayesian OptimisationReinforcement LearningDeep Interactive Bayesian Reinforcement Learning via Meta-Learning
Agents that interact with other agents often do not know a priori what the other agents' strategies are, but have to maximise their own online return while interacting with and learning about others. The optimal adaptive…
Meta-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)