Divide and Explore: Multi-Agent Separate Exploration with Shared Intrinsic Motivations
One of the greatest challenges of reinforcement learning is efficient exploration, especially when training signals are sparse or deceptive. The main difficulty of exploration lies in the size and complexity of the state space, which makes simple approaches such as exhaustive search infeasible. Our work is based on two important observations. On one hand, modern computing platforms are extremely scalable in terms of number of computing nodes and cores, which can complete asynchronous and well load-balanced computational tasks very fast. On the other hand, Divide-and-Conquer is a commonly used technique in computer science to solve similar problems (such as SAT) of doing efficient search in extremely large state space. In this paper, we apply the idea of divide-and-conquer in the context of intelligent exploration. The resulting exploration scheme can be combined with various specific intrinsic rewards designed for the given task. In our exploration scheme, the learning algorithm can automatically divide the state space into regions, and each agent is assigned to explore one of these regions. All the agents run asynchronously and they can be deployed onto modern distributed computing platforms. Our experiments show that the proposed method is highly efficient and is able to achieve state-of-the-art results in many RL tasks such as MiniGrid and Vizdoom.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed ComputingEfficient ExplorationSimilar Papers 제목 키워드 기반
Learning Intrinsically Motivated Options to Stimulate Policy Exploration
A Reinforcement Learning (RL) agent needs to find an optimal sequence of actions in order to maximize rewards. This requires consistent exploration of states and action sequences to ensure the policy found is optimal. On…
reinforcement-learningReinforcement Learning (RL)Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
Recent advancements in agentic test-time scaling allow models to gather environmental feedback before committing to final actions. A key limitation of existing methods is that they typically employ undifferentiated explo…
Reinforcement LearningIntrinsic Exploration as Multi-Objective RL
Intrinsic motivation enables reinforcement learning (RL) agents to explore when rewards are very sparse, where traditional exploration heuristics such as Boltzmann or e-greedy would typically fail. However, intrinsic exp…
continuous-controlContinuous ControlReinforcement LearningReinforcement Learning (RL)Explainable Multi-Modal Data Exploration in Natural Language via LLM Agent
International enterprises, organizations, or hospitals collect large amounts of multi-modal data stored in databases, text documents, images, and videos. While there has been recent progress in the separate fields of mul…
Text to SQLText-To-SQLMulti-agent Deep Covering Skill Discovery
The use of skills (a.k.a., options) can greatly accelerate exploration in reinforcement learning, especially when only sparse reward signals are available. While option discovery methods have been proposed for individual…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)