Language-Guided Reinforcement Learning for Hard Attention in Few-Shot Learning
Attention mechanisms have demonstrated significant potential in enhancing learning models by identifying key portions of input data, particularly in scenarios with limited training samples. Inspired by human perception, we propose that focusing on essential data segments, rather than the entire dataset, can improve the accuracy and reliability of the learning models. However, identifying these critical data segments, or "hard attention finding," is challenging, especially in few-shot learning, due to the scarcity of training data and the complexity of model parameters. To address this, we introduce LaHA, a novel framework that leverages language-guided deep reinforcement learning to identify and utilize informative data regions, thereby improving both interpretability and performance. Extensive experiments on benchmark datasets validate the effectiveness of LaHA.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningFew-Shot LearningHard AttentionSimilar Papers 제목 키워드 기반
Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance
Reinforcement Learning with Verifiable Rewards (RLVR) has achieved great success in developing Large Language Models (LLMs) with chain-of-thought rollouts for many tasks such as math and coding. Nevertheless, RLVR strugg…
Reinforcement LearningSim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving
Vision-language-model (VLM)-guided reinforcement learning (RL) has recently attracted significant attention for it, replacing brittle hand-crafted rewards with semantically grounded signals; however, deploying such simul…
Reinforcement LearningAutonomous DrivingA Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation
Increasing attention is being diverted to data-efficient problem settings like Open Vocabulary Semantic Segmentation (OVSS) which deals with segmenting an arbitrary object that may or may not be seen during training. The…
Few-Shot Semantic SegmentationLanguage ModellingOpen Vocabulary Semantic SegmentationOpen-Vocabulary Semantic Segmentation+3Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models
Due to the impressive zero-shot capabilities, pre-trained vision-language models (e.g. CLIP), have attracted widespread attention and adoption across various domains. Nonetheless, CLIP has been observed to be susceptible…
Adversarial RobustnessAllZero-shot Transfer of Reinforcement Learning Control Policies for the Swing-Up and Stabilization of a Cart-Pole System
Reinforcement learning (RL) is a powerful and convenient tool to modernize controller design. In this work, we study the zero-shot transfer of RL-based control policies from simulation to hardware for cart-pole swing-up …
Reinforcement Learning