Foundation Models for Semantic Novelty in Reinforcement Learning
Effectively exploring the environment is a key challenge in reinforcement learning (RL). We address this challenge by defining a novel intrinsic reward based on a foundation model, such as contrastive language image pretraining (CLIP), which can encode a wealth of domain-independent semantic visual-language knowledge about the world. Specifically, our intrinsic reward is defined based on pre-trained CLIP embeddings without any fine-tuning or learning on the target RL task. We demonstrate that CLIP-based intrinsic rewards can drive exploration towards semantically meaningful states and outperform state-of-the-art methods in challenging sparse-reward procedurally-generated environments.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Models
Exploration remains a significant challenge in reinforcement learning, especially in environments where extrinsic rewards are sparse or non-existent. The recent rise of foundation models, such as CLIP, offers an opportun…
reinforcement-learningReinforcement LearningFoundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments
The next generation of autonomous agents must not only learn efficiently but also act reliably and adapt their behavior in open worlds. Standard approaches typically assume fixed tasks and environments with little or no …
Reinforcement LearningProgram SynthesisOptimizing Novelty of Top-k Recommendations using Large Language Models and Reinforcement Learning
Given an input query, a recommendation model is trained using user feedback data (e.g., click data) to output a ranked list of items. In real-world systems, besides accuracy, an important consideration for a new model is…
Product RecommendationReinforcement Learning (RL)Emotion-Gradient Metacognitive RSI (Part I): Theoretical Foundations and Single-Agent Architecture
We present the Emotion-Gradient Metacognitive Recursive Self-Improvement (EG-MRSI) framework, a novel architecture that integrates introspective metacognition, emotion-based intrinsic motivation, and recursive self-modif…
InformativenessDetecting and Adapting to Novelty in Games
Open-world novelty occurs when the rules of an environment can change abruptly, such as when a game player encounters "house rules". To address open-world novelty, game playing agents must be able to detect when novelty …
Knowledge GraphsModel-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)