Reinforcement Learning Algorithm Selection
This paper formalises the problem of online algorithm selection in the context of Reinforcement Learning. The setup is as follows: given an episodic task and a finite number of off-policy RL algorithms, a meta-algorithm has to decide which RL algorithm is in control during the next episode so as to maximize the expected return. The article presents a novel meta-algorithm, called Epochal Stochastic Bandit Algorithm Selection (ESBAS). Its principle is to freeze the policy updates at each epoch, and to leave a rebooted stochastic bandit in charge of the algorithm selection. Under some assumptions, a thorough theoretical analysis demonstrates its near-optimality considering the structural sampling budget limitations. ESBAS is first empirically evaluated on a dialogue task where it is shown to outperform each individual algorithm in most configurations. ESBAS is then adapted to a true online setting where algorithms update their policies after each transition, which we call SSBAS. SSBAS is evaluated on a fruit collection task where it is shown to adapt the stepsize parameter more efficiently than the classical hyperbolic decay, and on an Atari game, where it improves the performance by a wide margin.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Learning Rate-Free Reinforcement Learning: A Case for Model Selection with Non-Stationary Objectives
The performance of reinforcement learning (RL) algorithms is sensitive to the choice of hyperparameters, with the learning rate being particularly influential. RL algorithms fail to reach convergence or demand an extensi…
Model Selectionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Tournament selection in zeroth-level classifier systems based on average reward reinforcement learning
As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) is based on a discounted reward reinforcement learning algorithm, bucket-brigade algorithm, which optimizes the discounted total reward…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)The Exploratory Multi-Asset Mean-Variance Portfolio Selection using Reinforcement Learning
In this paper, we study the continuous-time multi-asset mean-variance (MV) portfolio selection using a reinforcement learning (RL) algorithm, specifically the soft actor-critic (SAC) algorithm, in the time-varying financ…
Reinforcement Learning (RL)Reinforcement Learning Approach for Parallelization in Filters Aggregation Based Feature Selection Algorithms
One of the classical problems in machine learning and data mining is feature selection. A feature selection algorithm is expected to be quick, and at the same time it should show high performance. MeLiF algorithm effecti…
BIG-bench Machine Learningfeature selectionreinforcement-learningReinforcement Learning+1Enhancing Classification Performance via Reinforcement Learning for Feature Selection
Feature selection plays a crucial role in improving predictive accuracy by identifying relevant features while filtering out irrelevant ones. This study investigates the importance of effective feature selection in enhan…
Classificationfeature selectionQ-Learningreinforcement-learning+2