Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison

2023-10-06 · Moritz Lange, Noah Krystiniak, Raphael C. Engelhardt, Wolfgang Konen, Laurenz Wiskott

Real-world reinforcement learning (RL) environments, whether in robotics or industrial settings, often involve non-visual observations and require not only efficient but also reliable and thus interpretable and flexible RL approaches. To improve efficiency, agents that perform state representation learning with auxiliary tasks have been widely studied in visual observation contexts. However, for real-world problems, dedicated representation learning modules that are decoupled from RL agents are more suited to meet requirements. This study compares common auxiliary tasks based on, to the best of our knowledge, the only decoupled representation learning method for low-dimensional non-visual observations. We evaluate potential improvements in sample efficiency and returns for environments ranging from a simple pendulum to a complex simulated robotics task. Our findings show that representation learning with auxiliary tasks only provides performance gains in sufficiently complex environments and that learning environment dynamics is preferable to predicting rewards. These insights can inform future development of interpretable representation learning approaches for non-visual observations and advance the use of RL solutions in real-world scenarios.

📄 PDF Abstract BibTeX arXiv:2310.04241

Code (0)

등록된 구현이 없습니다.

Tasks

Continuous Control reinforcement-learning Reinforcement Learning (RL)Representation Learning

Adaptive Auxiliary Task Weighting for Reinforcement Learning

2019-12-01 · NeurIPS 2019 12 · Xingyu Lin, Harjatin Baweja, George Kantor, David Held

Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images. Transferring knowledge from other auxiliary tas…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning

2022-03-02 · Durgesh Kalwar, Omkar Shelke, Somjit Nath, Hardik Meisheri 외

Improving sample efficiency is a key challenge in reinforcement learning, especially in environments with large state spaces and sparse rewards. In literature, this is resolved either through the use of auxiliary tasks (…

reinforcement-learning Reinforcement Learning (RL)

Vision-based Navigation Using Deep Reinforcement Learning

2019-08-08 · Jonáš Kulhánek, Erik Derner, Tim de Bruin, Robert Babuška

Deep reinforcement learning (RL) has been successfully applied to a variety of game-like environments. However, the application of deep RL to visual navigation with realistic environments is a challenging task. We propos…

Deep Reinforcement Learning Efficient Neural Network reinforcement-learning Reinforcement Learning+2

Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays

2024-02-05 · Qingyuan Wu, Simon Sinong Zhan, YiXuan Wang, Yuhui Wang 외

Reinforcement learning (RL) is challenging in the common case of delays between events and their sensory perceptions. State-of-the-art (SOTA) state augmentation techniques either suffer from state space explosion or perf…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Auxiliary task discovery through generate-and-test

2022-10-25 · Banafsheh Rafiee, Sina Ghiassian, Jun Jin, Richard Sutton 외

In this paper, we explore an approach to auxiliary task discovery in reinforcement learning based on ideas from representation learning. Auxiliary tasks tend to improve data efficiency by forcing the agent to learn auxil…

Meta-Learning Representation Learning