Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning
In order for reinforcement learning (RL) agents to be deployed in real-world environments they must be able to generalize to unseen environments. However RL struggles with out-of-distribution generalization often due to over-fitting the particulars of the training environment. Although regularization techniques from supervised learning can be applied to avoid over-fitting the differences between supervised learning and RL limit their application. To address this we propose the Signal-to-Noise Ratio regulated Parameter Uncertainty Network (SNR PUN) for RL. We introduce SNR as a new measure of regularizing the parameter uncertainty of a network and provide a formal analysis explaining why SNR regularization works well for RL. We demonstrate the effectiveness of our proposed method to generalize in several simulated environments; and in a physical system showing the possibility of using SNR PUN for applying RL to real-world applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Out-of-Distribution Generalizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability
In the Bayesian reinforcement learning (RL) setting, a prior distribution over the unknown problem parameters -- the rewards and transitions -- is assumed, and a policy that optimizes the (posterior) expected return is s…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Uncertainty Regularized Multi-Task Learning
By sharing parameters and providing task-independent shared features, multi-task deep neural networks are considered one of the most interesting ways for parallel learning from different tasks and domains. However, fine-…
Multi-Task Learningtext-classificationText ClassificationTransfer LearningMURO: Deployment Constrained Reinforcement Learning with Model-based Uncertainty Regularized Batch Optimization
In many contemporary applications such as healthcare, finance, robotics, and recommendation systems, continuous deployment of new policies for data collection and online learning is either cost ineffective or impractical…
Recommendation Systemsreinforcement-learningReinforcement Learning (RL)Uncertainty QuantificationMUSBO: Model-based Uncertainty Regularized and Sample Efficient Batch Optimization for Deployment Constrained Reinforcement Learning
In many contemporary applications such as healthcare, finance, robotics, and recommendation systems, continuous deployment of new policies for data collection and online learning is either cost ineffective or impractical…
Reinforcement Learning (RL)Uncertainty QuantificationDoubly Robust Data-Driven Distributionally Robust Optimization
Data-driven Distributionally Robust Optimization (DD-DRO) via optimal transport has been shown to encompass a wide range of popular machine learning algorithms. The distributional uncertainty size is often shown to corre…