Prompting Decision Transformer for Few-Shot Policy Generalization
Humans can leverage prior experience and learn novel tasks from a handful of demonstrations. In contrast to offline meta-reinforcement learning, which aims to achieve quick adaptation through better algorithm design, we investigate the effect of architecture inductive bias on the few-shot learning capability. We propose a Prompt-based Decision Transformer (Prompt-DT), which leverages the sequential modeling ability of the Transformer architecture and the prompt framework to achieve few-shot adaptation in offline RL. We design the trajectory prompt, which contains segments of the few-shot demonstrations, and encodes task-specific information to guide policy generation. Our experiments in five MuJoCo control benchmarks show that Prompt-DT is a strong few-shot learner without any extra finetuning on unseen target tasks. Prompt-DT outperforms its variants and strong meta offline RL baselines by a large margin with a trajectory prompt containing only a few timesteps. Prompt-DT is also robust to prompt length changes and can generalize to out-of-distribution (OOD) environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningInductive BiasMeta Reinforcement LearningMuJoCoOffline RLMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance
Decision transformers recast reinforcement learning as a conditional sequence generation problem, offering a simple but effective alternative to traditional value or policy-based methods. A recent key development in this…
MuJoCoProD: Prompting-To-Disentangle Domain Knowledge for Cross-Domain Few-Shot Image Classification
This paper considers few-shot image classification under the cross-domain scenario, where the train-to-test domain gap compromises classification accuracy. To mitigate the domain gap, we propose a prompting-to-disent…
Cross-Domain Few-ShotDomain GeneralizationFew-Shot Image Classificationimage-classification+1A Robust PPO-optimized Tabular Transformer Framework for Intrusion Detection in Industrial IoT Systems
In this paper, we propose a robust and reinforcement-learning-enhanced network intrusion detection system (NIDS) designed for class-imbalanced and few-shot attack scenarios in Industrial Internet of Things (IIoT) environ…
Intrusion DetectionNetwork Intrusion Detectionreinforcement-learningReinforcement LearningA Minimalist Prompt for Zero-Shot Policy Learning
Transformer-based methods have exhibited significant generalization ability when prompted with target-domain demonstrations or example solutions during inference. Although demonstrations, as a way of task specification, …
Zero-shot GeneralizationVisual Prompting for Generalized Few-shot Segmentation: A Multi-scale Approach
The emergence of attention-based transformer models has led to their extensive use in various tasks, due to their superior generalization and transfer properties. Recent research has demonstrated that such models, when p…
DecoderGeneralized Few-Shot Semantic SegmentationSemantic SegmentationVisual Prompting