On The Effect of Auxiliary Tasks on Representation Dynamics

2021-02-25 · Clare Lyle, Mark Rowland, Georg Ostrovski, Will Dabney

While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understanding of the relationship between auxiliary tasks, environment structure, and representations by analysing the dynamics of temporal difference algorithms. Through this approach, we establish a connection between the spectral decomposition of the transition operator and the representations induced by a variety of auxiliary tasks. We then leverage insights from these theoretical results to inform the selection of auxiliary tasks for deep reinforcement learning agents in sparse-reward environments.

📄 PDF Abstract BibTeX arXiv:2102.13089

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Why and How Auxiliary Tasks Improve JEPA Representations

2025-09-12 · Jiacan Yu, Siyi Chen, Mingrui Liu, Nono Horiuchi 외 arxiv

Joint-Embedding Predictive Architecture (JEPA) is increasingly used for visual representation learning and as a component in model-based RL, but its behavior remains poorly understood. We provide a theoretical characteri…

Representation Learning

Evaluating Factor-Wise Auxiliary Dynamics Supervision for Latent Structure and Robustness in Simulated Humanoid Locomotion

2026-03-22 · Chayanin Chamachot arxiv

We evaluate whether factor-wise auxiliary dynamics supervision produces useful latent structure or improved robustness in simulated humanoid locomotion. DynaMITE -- a transformer encoder with a factored 24-d latent train…

Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison

2023-10-06 · Moritz Lange, Noah Krystiniak, Raphael C. Engelhardt, Wolfgang Konen 외

Real-world reinforcement learning (RL) environments, whether in robotics or industrial settings, often involve non-visual observations and require not only efficient but also reliable and thus interpretable and flexible …

Continuous Control reinforcement-learning Reinforcement Learning (RL)Representation Learning

When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning

2024-06-25 · Claas Voelcker, Tyler Kastner, Igor Gilitschenski, Amir-Massoud Farahmand

We investigate the impact of auxiliary learning tasks such as observation reconstruction and latent self-prediction on the representation learning problem in reinforcement learning. We also study how they interact with d…

Auxiliary Learning Prediction reinforcement-learning Representation Learning

Factorized Latent Dynamics for Video JEPA: An Empirical Study of Auxiliary Objectives

2026-05-16 · Santosh Premi arxiv

Joint-Embedding Predictive Architectures (JEPA) are a promising framework for self-supervised video representation learning, yet the behavior of auxiliary objectives in small-scale Video-JEPA training is not well charact…