Relevant Action Matters : Motivating agent with action usefulness
Sparse rewards are double-edged training signals in reinforcement learning: easy to design but hard to optimize. Intrinsic motivation guidances have thus been developed toward alleviating the resulting exploration problem. They usually incentivize agents to look for new states through novelty signals. Yet, such methods encourage exhaustive exploration of the state space rather than focusing on the environment's salient interaction opportunities. We propose a new exploration method, called Relevant Action Matters (RAM), shifting the emphasis from state novelty to state with relevant actions. While most actions consistently change the state when used, e.g. moving the agent, some actions are only effective in specific states, e.g., opening a door, grabbing an object. RAM detects and rewards actions that seldom affect the environment. We evaluate RAM on the procedurally-generated environment MiniGrid, against state-of-the-art methods. Experiments consistently show that RAM greatly reduces sample complexity, installing the new state-of-the-art in MiniGrid.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
In long-horizon tasks, decision-relevant state is often scattered across an expanding trajectory, while the action agent must surface it and act. As trajectories grow, task requirements, environment facts, prior attempts…
Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning
Reinforcement Learning in large action spaces is a challenging problem. Cooperative multi-agent reinforcement learning (MARL) exacerbates matters by imposing various constraints on communication and observability. In thi…
Learning TheoryMulti-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+2Safe, or Simply Incapable? Rethinking Safety Evaluation for Phone-Use Agents
When a phone-use agent avoids harm, does that show safety, or simply inability to act? Existing evaluations often cannot tell. A harmful outcome may be avoided because the agent recognized the risk and chose the safe act…
LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action
Vision-language-action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual tokens uniformly and reasoning w…
SANA: What Matters for QA Agents over Massive Data Lakes?
Exploratory question answering (EQA) over data lakes requires an LLM agent to discover relevant sources, analyze retrieved data, and adapt its actions based on intermediate results. End-to-end accuracy alone cannot disti…
Question Answering