Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning

2019-09-09 · Arjun Manoharan, Rahul Ramesh, Balaraman Ravindran

Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent. However, such techniques often yield a large number of options or skills, which can potentially be represented succinctly by filtering out any redundant information. Such a reduction can reduce the required computation while also improving the performance on a target task. In order to compress an array of option policies, we attempt to find a policy basis that accurately captures the set of all options. In this work, we propose Option Encoder, an auto-encoder based framework with intelligently constrained weights, that helps discover a collection of basis policies. The policy basis can be used as a proxy for the original set of skills in a suitable hierarchically organized framework. We demonstrate the efficacy of our method on a collection of grid-worlds and on the high-dimensional Fetch-Reach robotic manipulation task by evaluating the obtained policy basis on a set of downstream tasks.

📄 PDF Abstract BibTeX arXiv:1909.04134

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

The Eigenoption-Critic Framework

2017-12-11 · Miao Liu, Marlos C. Machado, Gerald Tesauro, Murray Campbell

Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration. Despite its initial promising re…

Efficient Exploration Hierarchical Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Context-Specific Representation Abstraction for Deep Option Learning

2021-09-20 · Marwa Abdulhai, Dong-Ki Kim, Matthew Riemer, Miao Liu 외

Hierarchical reinforcement learning has focused on discovering temporally extended actions, such as options, that can provide benefits in problems requiring extensive exploration. One promising approach that learns these…

Hierarchical Reinforcement Learning

Discovery of Options via Meta-Learned Subgoals

2021-02-12 · NeurIPS 2021 12 · Vivek Veeriah, Tom Zahavy, Matteo Hessel, Zhongwen Xu 외

Temporal abstractions in the form of options have been shown to help reinforcement learning (RL) agents learn faster. However, despite prior work on this topic, the problem of discovering options through interaction with…

Reinforcement Learning (RL)

Iterative Option Discovery for Planning, by Planning

2023-10-02 · Kenny Young, Richard S. Sutton

Discovering useful temporal abstractions, in the form of options, is widely thought to be key to applying reinforcement learning and planning to increasingly complex domains. Building on the empirical success of the Expe…

Discovering Sparse Counterfactual Factors via Latent Adjustment for Survey-based Community Intervention

2026-05-06 · Fatima Ashraf, Muhammad Ayub Sabir, Junbiao Pang, Yufang Zhou 외 arxiv

Transportation surveys are widely used to understand travel preferences and adoption barriers, yet most survey-based analyses remain descriptive or predictive and rarely provide sparse, policy-feasible intervention strat…