홈 › Papers

MO2: Model-Based Offline Options

2022-09-05 · Sasha Salter, Markus Wulfmeier, Dhruva Tirumala, Nicolas Heess, Martin Riedmiller, Raia Hadsell, Dushyant Rao

The ability to discover useful behaviours from past experience and transfer them to new tasks is considered a core component of natural embodied intelligence. Inspired by neuroscience, discovering behaviours that switch at bottleneck states have been long sought after for inducing plans of minimum description length across tasks. Prior approaches have either only supported online, on-policy, bottleneck state discovery, limiting sample-efficiency, or discrete state-action domains, restricting applicability. To address this, we introduce Model-Based Offline Options (MO2), an offline hindsight framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces. Once bottleneck options are learnt offline over source domains, they are transferred online to improve exploration and value estimation on the transfer domain. Our experiments show that on complex long-horizon continuous control tasks with sparse, delayed rewards, MO2's properties are essential and lead to performance exceeding recent option learning methods. Additional ablations further demonstrate the impact on option predictability and credit assignment.

📄 PDF Abstract BibTeX arXiv:2209.01947

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Controlmodel

Provably (More) Sample-Efficient Offline RL with Options

2023-09-21 · NeurIPS 2023 11

The options framework yields empirical success in long-horizon planning problems of reinforcement learning (RL). Recent works show that options help improve the sample efficiency in online RL. However, these results are …

Temporal Abstraction in Reinforcement Learning with Offline Data

2024-07-21 · Ranga Shaarad Ayyagari, Anurita Ghosh, Ambedkar Dukkipati

Standard reinforcement learning algorithms with a single policy perform poorly on tasks in complex environments involving sparse rewards, diverse behaviors, or long-term planning. This led to the study of algorithms that…

Hierarchical Reinforcement Learning MuJoCo reinforcement-learning Reinforcement Learning

Abstraction for Offline Goal-Conditioned Reinforcement Learning

2026-05-21 · Clarisse Wibault, Alexander Goldie, Antonio Villares, Maike Osborne 외 arxiv

Markov Decision Processes (MDPs) often exhibit significant redundancy due to symmetries and shared structure across state-goal pairs in real-world Goal-Conditioned Reinforcement Learning (GCRL). While hierarchical polici…

Reinforcement Learning

Bayesian Nonparametrics for Offline Skill Discovery

2022-02-09 · Valentin Villecroze, Harry J. Braviner, Panteha Naderian, Chris J. Maddison 외

Skills or low-level policies in reinforcement learning are temporally extended actions that can speed up learning and enable complex behaviours. Recent work in offline reinforcement learning and imitation learning has pr…

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

On Calibration Neural Networks for extracting implied information from American options

2020-01-31 · Shuaiqiang Liu, Álvaro Leitao, Anastasia Borovykh, Cornelis W. Oosterlee

Extracting implied information, like volatility and/or dividend, from observed option prices is a challenging task when dealing with American options, because of the computational costs needed to solve the corresponding …

BIG-bench Machine Learning