Task diversity produces systematic transfer but inhibits continual reinforcement learning
Continual reinforcement learning aims to produce agents that learn not only to improve at their current tasks but also to adapt as task distributions change. Training an agent on many diverse tasks can induce zero-shot generalization, but previous work generally evaluates this generalization after training -- with frozen weights. Whether task diversity also improves an agent's ability to continue learning across distribution shifts remains unclear. We introduce Banyan, a GPU-accelerated continual RL domain in which task diversity factors into three independently controllable axes: the map layouts an agent must navigate, the objects it must interact with, and the hierarchical structures of sub-goal dependencies. Across individual distribution shifts, increasing diversity along each axis causes agents to begin training on the new tasks near the performance attained on the previous one, even when the shift changes the structure of the optimal policy. However, as the number of shifts increases, this local transfer does not by itself yield sustained continual learning: longer-horizon tasks plateau, and earlier task distributions are forgotten after later training. Banyan is a benchmark for studying when controlled task diversity produces transferable learning, when that transfer persists, and where it falls short of proper continual learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Zero-shot GeneralizationReinforcement LearningContinual LearningSimilar Papers 제목 키워드 기반
Quality Diversity for Visual Pre-Training
Models pre-trained on large datasets such as ImageNet provide the de-facto standard for transfer learning, with both supervised and self-supervised approaches proving effective. However, emerging evidence suggests th…
DiversityInductive BiasTransfer LearningThe impact of data set similarity and diversity on transfer learning success in time series forecasting
Pre-trained models have become pivotal in enhancing the efficiency and accuracy of time series forecasting on target data sets by leveraging transfer learning. While benchmarks validate the performance of model generaliz…
DiversityTime SeriesTime Series ForecastingTransfer LearningEmpirical Evaluation of Supervision Signals for Style Transfer Models
Text style transfer has gained increasing attention from the research community over the recent years. However, the proposed approaches vary in many ways, which makes it hard to assess the individual contribution of the …
Machine Translationreinforcement-learningReinforcement LearningReinforcement Learning (RL)+3Data Analogies Enable Efficient Cross-Embodiment Transfer
Generalist robot policies are trained on demonstrations collected across a wide variety of robots, scenes, and viewpoints. Yet it remains unclear how to best organize and scale such heterogeneous data so that it genuinel…
Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation
Weakly supervised semantic segmentation produces a pixel-level localization from a classifier, but it is likely to restrict its focus to a small discriminative region of the target object. AdvCAM is an attribution map of…
Adversarial AttackObjectSegmentationSemantic Segmentation+3