paper-with-me

홈 › Papers

Meta-Reinforcement Learning for the Tuning of PI Controllers: An Offline Approach

2022-03-17 · Daniel G. McClement, Nathan P. Lawrence, Johan U. Backstrom, Philip D. Loewen, Michael G. Forbes, R. Bhushan Gopaluni

Meta-learning is a branch of machine learning which trains neural network models to synthesize a wide variety of data in order to rapidly solve new problems. In process control, many systems have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, we formulate a meta reinforcement learning (meta-RL) control strategy that can be used to tune proportional--integral controllers. Our meta-RL agent has a recurrent structure that accumulates "context" to learn a system's dynamics through a hidden state variable in closed-loop. This architecture enables the agent to automatically adapt to changes in the process dynamics. In tests reported here, the meta-RL agent was trained entirely offline on first order plus time delay systems, and produced excellent results on novel systems drawn from the same distribution of process dynamics used for training. A key design element is the ability to leverage model-based information offline during training in simulated environments while maintaining a model-free policy structure for interacting with novel processes where there is uncertainty regarding the true process dynamics. Meta-learning is a promising approach for constructing sample-efficient intelligent controllers.

📄 PDF Abstract BibTeX arXiv:2203.09661

Code (0)

등록된 구현이 없습니다.

Tasks

Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning (RL)

Similar Papers 제목 키워드 기반

Meta-Reinforcement Learning for Adaptive Control of Second Order Systems

2022-09-19 · Daniel G. McClement, Nathan P. Lawrence, Michael G. Forbes, Philip D. Loewen 외

Meta-learning is a branch of machine learning which aims to synthesize data from a distribution of related tasks to efficiently solve new ones. In process control, many systems have similar and well-understood dynamics, …

Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1

Contextual Transformer for Offline Meta Reinforcement Learning

2022-11-15 · Runji Lin, Ye Li, Xidong Feng, Zhaowei Zhang 외

The pretrain-finetuning paradigm in large-scale sequence models has made significant progress in natural language processing and computer vision tasks. However, such a paradigm is still hindered by several challenges in …

D4RLMeta Reinforcement LearningMuJoCoOffline RL+5

Offline Meta-Reinforcement Learning with Advantage Weighting

2020-08-13 · Eric Mitchell, Rafael Rafailov, Xue Bin Peng, Sergey Levine 외

This paper introduces the offline meta-reinforcement learning (offline meta-RL) problem setting and proposes an algorithm that performs well in this setting. Offline meta-RL is analogous to the widely successful supervis…

Machine TranslationMeta-LearningMeta Reinforcement LearningObject Recognition+4

Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers

2026-04-14 · Snehal Jauhri, Vignesh Prasad, Georgia Chalvatzaki arxiv

Mobile Manipulation (MoMa) of articulated objects, such as opening doors, drawers, and cupboards, demands simultaneous, whole-body coordination between a robot's base and arms. Classical whole-body controllers (WBCs) can…

Reinforcement LearningOffline RL

Offline Hierarchical Reinforcement Learning via Inverse Optimization

2024-10-10 · Carolin Schmidt, Daniele Gammelli, James Harrison, Marco Pavone 외

Hierarchical policies enable strong performance in many sequential decision-making problems, such as those with high-dimensional action spaces, those requiring long-horizon planning, and settings with sparse rewards. How…

Decision MakingHierarchical Reinforcement Learningreinforcement-learningReinforcement Learning+2