When should agents explore?
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradually (at best). In contrast, the exploratory behaviours of animals and humans exhibit a rich diversity, namely including forms of switching between modes. This paper presents an initial study of mode-switching, non-monolithic exploration for RL. We investigate different modes to switch between, at what timescales it makes sense to switch, and what signals make for good switching triggers. We also propose practical algorithmic components that make the switching mechanism adaptive and robust, which enables flexibility without an accompanying hyper-parameter-tuning burden. Finally, we report a promising and detailed analysis on Atari, using two-mode exploration and switching at sub-episodic time-scales.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Aggregating incoherent agents who disagree
In this paper, we explore how we should aggregate the degrees of belief of of a group of agents to give a single coherent set of degrees of belief, when at least some of those agents might be probabilistically incoherent…
When Should Agents Coordinate in Differentiable Sequential Decision Problems?
Multi-robot teams must coordinate to operate effectively. When a team operates in an uncoordinated manner, and agents choose actions that are only individually optimal, the team's outcome can suffer. However, in many dom…
DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning
Learning optimal behavior policy for each agent in multi-agent systems is an essential yet difficult problem. Despite fruitful progress in multi-agent reinforcement learning, the challenge of addressing the dynamics of w…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningStarcraft+1Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback
Many approaches to Natural Language Processing (NLP) tasks often treat them as single-step problems, where an agent receives an instruction, executes it, and is evaluated based on the final outcome. However, human langua…
AI AgentMinecraftThe Automated but Risky Game: Modeling Agent-to-Agent Negotiations and Transactions in Consumer Markets
AI agents are increasingly used in consumer-facing applications to assist with tasks such as product search, negotiation, and transaction execution. In this paper, we explore a future scenario where both consumers and me…