Exploration and Incentives in Reinforcement Learning

2021-02-28 · Max Simchowitz, Aleksandrs Slivkins

How do you incentivize self-interested agents to $\textit{explore}$ when they prefer to $\textit{exploit}$? We consider complex exploration problems, where each agent faces the same (but unknown) MDP. In contrast with traditional formulations of reinforcement learning, agents control the choice of policies, whereas an algorithm can only issue recommendations. However, the algorithm controls the flow of information, and can incentivize the agents to explore via information asymmetry. We design an algorithm which explores all reachable states in the MDP. We achieve provable guarantees similar to those for incentivizing exploration in static, stateless exploration problems studied previously. To the best of our knowledge, this is the first work to consider mechanism design in a stateful, reinforcement learning setting.

📄 PDF Abstract BibTeX arXiv:2103.00360

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Intrinsically-Motivated Reinforcement Learning: A Brief Introduction

2022-03-03 · Mingqi Yuan

Reinforcement learning (RL) is one of the three basic paradigms of machine learning. It has demonstrated impressive performance in many complex tasks like Go and StarCraft, which is increasingly involved in smart manufac…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Exploration with Unreliable Intrinsic Reward in Multi-Agent Reinforcement Learning

2019-06-05 · Wendelin Böhmer, Tabish Rashid, Shimon Whiteson

This paper investigates the use of intrinsic reward to guide exploration in multi-agent reinforcement learning. We discuss the challenges in applying intrinsic reward to multiple collaborative agents and demonstrate how …

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning

2023-01-26 · Mingqi Yuan, Bo Li, Xin Jin, Wenjun Zeng

We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and adaptively provides high-quality intrinsic rewards to enhance exploration in reinforcement learning (RL). More specifically, AIRS selects shaping…

Benchmarking Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL

2022-08-22 · Andreas A. Haupt, Phillip J. K. Christoffersen, Mehul Damani, Dylan Hadfield-Menell

Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group …

Management Multi-agent Reinforcement Learning Multi-Objective Reinforcement Learning reinforcement-learning

Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning

2017-03-06 · Joshua Achiam, Shankar Sastry

Exploration in complex domains is a key challenge in reinforcement learning, especially for tasks with very sparse rewards. Recent successes in deep reinforcement learning have been achieved mostly using simple heuristic…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2