Hierarchical Optimistic Region Selection driven by Curiosity

2012-12-01 · NeurIPS 2012 12 · Odalric-Ambrym Maillard

This paper aims to take a step forwards making the term ``intrinsic motivation'' from reinforcement learning theoretically well founded, focusing on curiosity-driven learning. To that end, we consider the setting where, a fixed partition P of a continuous space X being given, and a process \nu defined on X being unknown, we are asked to sequentially decide which cell of the partition to select as well as where to sample \nu in that cell, in order to minimize a loss function that is inspired from previous work on curiosity-driven learning. The loss on each cell consists of one term measuring a simple worst case quadratic sampling error, and a penalty term proportional to the range of the variance in that cell. The corresponding problem formulation extends the setting known as active learning for multi-armed bandits to the case when each arm is a continuous region, and we show how an adaptation of recent algorithms for that problem and of hierarchical optimistic sampling algorithms for optimization can be used in order to solve this problem. The resulting procedure, called Hierarchical Optimistic Region SElection driven by Curiosity (HORSE.C) is provided together with a finite-time regret analysis.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Active Learning Multi-Armed Bandits Reinforcement Learning

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

2022-09-15 · Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma 외

In this work, we study the simple yet universally applicable case of reward shaping in value-based Deep Reinforcement Learning (DRL). We show that reward shifting in the form of the linear transformation is equivalent to…

continuous-control Continuous Control Deep Reinforcement Learning Offline RL

Curious Hierarchical Actor-Critic Reinforcement Learning

2020-05-07 · Frank Röder, Manfred Eppe, Phuong D. H. Nguyen, Stefan Wermter

Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward spar…

Benchmarking Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reward Shifting for Optimistic Exploration and Conservative Exploitation

2021-09-29 · Hao Sun, Lei Han, Jian Guo, Bolei Zhou

In this work, we study the simple yet universally applicable case of reward shaping, the linear transformation, in value-based Deep Reinforcement Learning. We show that reward shifting, as the simplest linear reward tran…

continuous-control Continuous Control Deep Reinforcement Learning Offline RL

Curriculum-guided Hindsight Experience Replay

2019-12-01 · NeurIPS 2019 12 · Meng Fang, Tianyi Zhou, Yali Du, Lei Han 외

In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by t…

Deep Reinforcement Learning Diversity Reinforcement Learning

ACDER: Augmented Curiosity-Driven Experience Replay

2020-11-16 · Boyao Li, Tao Lu, Jiayi Li, Ning Lu 외

Exploration in environments with sparse feedback remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, espec…

FetchPush-v1 Reinforcement Learning (RL)