LLM-Driven Intrinsic Motivation for Sparse Reward Reinforcement Learning

2025-08-25 · André Quadros, Cassio Silva, Ronnie Alves arxiv

This paper explores the combination of two intrinsic motivation strategies to improve the efficiency of reinforcement learning (RL) agents in environments with extreme sparse rewards, where traditional learning struggles due to infrequent positive feedback. We propose integrating Variational State as Intrinsic Reward (VSIMR), which uses Variational AutoEncoders (VAEs) to reward state novelty, with an intrinsic reward approach derived from Large Language Models (LLMs). The LLMs leverage their pre-trained knowledge to generate reward signals based on environment and goal descriptions, guiding the agent. We implemented this combined approach with an Actor-Critic (A2C) agent in the MiniGrid DoorKey environment, a benchmark for sparse rewards. Our empirical results show that this combined strategy significantly increases agent performance and sampling efficiency compared to using each strategy individually or a standard A2C agent, which failed to learn. Analysis of learning curves indicates that the combination effectively complements different aspects of the environment and task: VSIMR drives exploration of new states, while the LLM-derived rewards facilitate progressive exploitation towards goals.

📄 PDF Abstract BibTeX arXiv:2508.18420

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Curiosity-driven Exploration in Sparse-reward Multi-agent Reinforcement Learning

2023-02-21 · Jiong Li, Pratik Gajane

Sparsity of rewards while applying a deep reinforcement learning method negatively affects its sample-efficiency. A viable solution to deal with the sparsity of rewards is to learn via intrinsic motivation which advocate…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

An Empowerment-based Solution to Robotic Manipulation Tasks with Sparse Rewards

2020-10-15 · Siyu Dai, Wei Xu, Andreas Hofmann, Brian Williams

In order to provide adaptive and user-friendly solutions to robotic manipulation, it is important that the agent can learn to accomplish tasks even if they are only provided with very sparse instruction signals. To addre…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Control

2022-11-28 · Xiang Zheng, Xingjun Ma, Cong Wang

Intrinsic motivation is a promising exploration technique for solving reinforcement learning tasks with sparse or absent extrinsic rewards. There exist two technical challenges in implementing intrinsic motivation: 1) ho…

continuous-control Continuous Control Efficient Exploration

PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillation

2024-10-02 · Mohammadamin Davoodabadi, Negin Hashemi Dijujin, Mahdieh Soleymani Baghshah

Intrinsic motivation, inspired by the psychology of developmental learning in infants, stimulates exploration in agents without relying solely on sparse external rewards. Existing methods in reinforcement learning like R…

Developmental Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning

2017-03-06 · Joshua Achiam, Shankar Sastry

Exploration in complex domains is a key challenge in reinforcement learning, especially for tasks with very sparse rewards. Recent successes in deep reinforcement learning have been achieved mostly using simple heuristic…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2