Model Embedding Model-Based Reinforcement Learning

2020-06-16 · Xiaoyu Tan, Chao Qu, Junwu Xiong, James Zhang

Model-based reinforcement learning (MBRL) has shown its advantages in sample-efficiency over model-free reinforcement learning (MFRL). Despite the impressive results it achieves, it still faces a trade-off between the ease of data generation and model bias. In this paper, we propose a simple and elegant model-embedding model-based reinforcement learning (MEMB) algorithm in the framework of the probabilistic reinforcement learning. To balance the sample-efficiency and model bias, we exploit both real and imaginary data in the training. In particular, we embed the model in the policy update and learn $Q$ and $V$ functions from the real data set. We provide the theoretical analysis of MEMB with the Lipschitz continuity assumption on the model and policy. At last, we evaluate MEMB on several benchmarks and demonstrate our algorithm can achieve state-of-the-art performance.

📄 PDF Abstract BibTeX arXiv:2006.09234

Code (0)

등록된 구현이 없습니다.

Tasks

modelModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Disentangled Skill Embeddings for Reinforcement Learning

2019-06-21 · Janith C. Petangoda, Sergio Pascual-Diaz, Vincent Adam, Peter Vrancx 외

We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamics and goals. The resulting policies are …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Jointly-Learned State-Action Embedding for Efficient Reinforcement Learning

2020-10-09 · Paul J. Pritz, Liang Ma, Kin K. Leung

While reinforcement learning has achieved considerable successes in recent years, state-of-the-art models are often still limited by the size of state and action spaces. Model-free reinforcement learning approaches use s…

Model-based Reinforcement Learning Recommendation Systems reinforcement-learning Reinforcement Learning+1

Plan-Space State Embeddings for Improved Reinforcement Learning

2020-04-30 · Max Pflueger, Gaurav S. Sukhatme

Robot control problems are often structured with a policy function that maps state values into control values, but in many dynamic problems the observed state can have a difficult to characterize relationship with useful…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Jointly-Trained State-Action Embedding for Efficient Reinforcement Learning

2020-09-28 · Paul Julian Pritz, Liang Ma, Kin Leung

Model-based Reinforcement Learning Recommendation Systems reinforcement-learning Reinforcement Learning+1

Manifold Embeddings for Model-Based Reinforcement Learning under Partial Observability

2009-12-01 · NeurIPS 2009 12 · Keith Bush, Joelle Pineau

Interesting real-world datasets often exhibit nonlinear, noisy, continuous-valued states that are unexplorable, are poorly described by first principles, and are only partially observable. If partial observability can be…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)