A Narration-based Reward Shaping Approach using Grounded Natural Language Commands
While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of reward sparsity. This is especially true for tasks such as training an agent to play StarCraft II, a real-time strategy game where reward is only given at the end of a game which is usually very long. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we develop a technique that can provide the benefits of reward shaping using natural language commands. Our narration-guided RL agent projects sequences of natural-language commands into the same high-dimensional representation space as corresponding goal states. We show that we can get improved performance with our method compared to traditional reward-shaping approaches. Additionally, we demonstrate the ability of our method to generalize to unseen natural-language commands.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningReinforcement LearningStarcraftStarcraft IISimilar Papers 제목 키워드 기반
Grounding Natural Language Commands to StarCraft II Game States for Narration-Guided Reinforcement Learning
While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longsta…
Deep Reinforcement LearningReinforcement LearningReinforcement Learning (RL)Starcraft+1VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping
In social impact optimization, AI decision systems often rely on solvers that optimize well-calibrated mathematical objectives. However, these solvers cannot directly accommodate evolving human preferences, typically exp…
Reward Learning from Narrated Demonstrations
Humans effortlessly "program" one another by communicating goals and desires in natural language. In contrast, humans program robotic behaviours by indicating desired object locations and poses to be achieved, by providi…
Using Natural Language for Reward Shaping in Reinforcement Learning
Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environme…
Atari GamesMontezuma's Revengereinforcement-learningReinforcement Learning+1Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a…
MuJoCoreinforcement-learningReinforcement LearningReinforcement Learning (RL)