Meta-Reinforcement Learning for Adaptive Control of Second Order Systems
Meta-learning is a branch of machine learning which aims to synthesize data from a distribution of related tasks to efficiently solve new ones. In process control, many systems have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, we formulate a meta reinforcement learning (meta-RL) control strategy that takes advantage of known, offline information for training, such as a model structure. The meta-RL agent is trained over a distribution of model parameters, rather than a single model, enabling the agent to automatically adapt to changes in the process dynamics while maintaining performance. A key design element is the ability to leverage model-based information offline during training, while maintaining a model-free policy structure for interacting with new environments. Our previous work has demonstrated how this approach can be applied to the industrially-relevant problem of tuning proportional-integral controllers to control first order processes. In this work, we briefly reintroduce our methodology and demonstrate how it can be extended to proportional-integral-derivative controllers and second order systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation
Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents …
Computational EfficiencyReinforcement LearningA Better Baseline for Second Order Gradient Estimation in Stochastic Computation Graphs
Motivated by the need for higher order gradients in multi-agent reinforcement learning and meta-learning, this paper studies the construction of baselines for second order Monte Carlo gradient estimators in order to redu…
Meta-LearningMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Meta-Learning-Based Delayless Subband Adaptive Filter using Complex Self-Attention for Active Noise Control
Active noise control typically employs adaptive filtering to generate secondary noise, where the least mean square algorithm is the most widely used. However, traditional updating rules are linear and exhibit limited eff…
Meta-LearningLearning Generalizable Representations for Reinforcement Learning via Adaptive Meta-learner of Behavioral Similarities
How to learn an effective reinforcement learning-based model for control tasks from high-level visual observations is a practical and challenging problem. A key to solving this problem is to learn low-dimensional state r…
Data Augmentationreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Complementary Meta-Reinforcement Learning for Fault-Adaptive Control
Faults are endemic to all systems. Adaptive fault-tolerant control maintains degraded performance when faults occur as opposed to unsafe conditions or catastrophic events. In systems with abrupt faults and strict time co…
Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1