Offline-Online Reinforcement Learning: Extending Batch and Online RL

2021-09-29 · Maryam Hashemzadeh, Wesley Chung, Martha White

Batch RL has seen a surge in popularity and is applicable in many practical scenarios where past data is available. Unfortunately, the performance of batch RL agents is limited in both theory and practice without strong assumptions on the data-collection process e.g. sufficient coverage or a good policy. To enable better performance, we investigate the offline-online setting: The agent has access to a batch of data to train on but is also allowed to learn during the evaluation phase in an online manner. This is an extension to batch RL, allowing the agent to adapt to new situations without having to precommit to a policy. In our experiments, we find that agents trained in an offline-online manner can outperform agents trained only offline or online, sometimes by a large margin, for different dataset sizes and data-collection policies. Furthermore, we investigate the use of optimism vs. pessimism for value functions in the offline-online setting due to their use in batch and online RL.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Offline versus Online Triplet Mining based on Extreme Distances of Histopathology Patches

2020-07-04 · Milad Sikaroudi, Benyamin Ghojogh, Amir Safarpoor, Fakhri Karray 외

We analyze the effect of offline and online triplet mining for colorectal cancer (CRC) histopathology dataset containing 100,000 patches. We consider the extreme, i.e., farthest and nearest patches to a given anchor, bot…

Dimensionality Reduction Histopathological Image Classification Metric Learning Triplet

Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems

2022-08-09 · Qihua Zhang, Junning Liu, Yuzhuo Dai, Yiyan Qi 외

Recommender System (RS) is an important online application that affects billions of users every day. The mainstream RS ranking framework is composed of two parts: a Multi-Task Learning model (MTL) that predicts various u…

Multi-Task Learning Recommendation Systems Reinforcement Learning (RL)

Striving for Simplicity in Off-Policy Deep Reinforcement Learning

2019-09-25 · Rishabh Agarwal, Dale Schuurmans, Mohammad Norouzi

This paper advocates the use of offline (batch) reinforcement learning (RL) to help (1) isolate the contributions of exploitation vs. exploration in off-policy deep RL, (2) improve reproducibility of deep RL research, an…

Atari Games Deep Reinforcement Learning Offline RL Q-Learning+3

Byzantine-Robust Online and Offline Distributed Reinforcement Learning

2022-06-01 · Yiding Chen, Xuezhou Zhang, Kaiqing Zhang, Mengdi Wang 외

We consider a distributed reinforcement learning setting where multiple agents separately explore the environment and communicate their experiences through a central server. However, $\alpha$-fraction of agents are adver…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Benchmarking Sample Selection Strategies for Batch Reinforcement Learning

2021-09-29 · Yuwei Fu, Di wu, Benoit Boulet

Training sample section techniques, such as prioritized experience replay (PER), have been recognized as of significant importance for online reinforcement learning algorithms. Efficient sample selection can help further…

Benchmarking Imitation Learning reinforcement-learning Reinforcement Learning+1