Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback
We consider reinforcement learning and bandit structured prediction problems with very sparse loss feedback: only at the end of an episode. We introduce a novel algorithm, RESIDUAL LOSS PREDICTION (RESLOPE), that solves such problems by automatically learning an internal representation of a denser reward function. RESLOPE operates as a reduction to contextual bandits, using its learned loss representation to solve the credit assignment problem, and a contextual bandit oracle to trade-off exploration and exploitation. RESLOPE enjoys a no-regret reduction-style theoretical guarantee and outperforms state of the art reinforcement learning algorithms in both MDP environments and bandit structured prediction settings.
Code (1)
Tasks
Multi-Armed BanditsPredictionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Structured PredictionSimilar Papers 제목 키워드 기반
Background Adaptation with Residual Modeling for Exemplar-Free Class-Incremental Semantic Segmentation
Class Incremental Semantic Segmentation~(CISS), within Incremental Learning for semantic segmentation, targets segmenting new categories while reducing the catastrophic forgetting on the old categories.Besides, backgroun…
Class-Incremental Semantic SegmentationExemplar-FreeIncremental LearningKnowledge Distillation+2Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control
The learning inefficiency of reinforcement learning (RL) from scratch hinders its practical application towards continuous robotic tracking control, especially for high-dimensional robots. This work proposes a data-infor…
reinforcement-learningReinforcement Learning (RL)Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation
As the demand for mobile robots continues to increase, social navigation has emerged as a critical task, driving active research into deep reinforcement learning (RL) approaches. However, because pedestrian dynamics and …
Reinforcement LearningIncremental LearningTwo-Level Residual Distillation based Triple Network for Incremental Object Detection
Modern object detection methods based on convolutional neural network suffer from severe catastrophic forgetting in learning new classes without original data. Due to time consumption, storage burden and privacy of old d…
Incremental LearningObjectobject-detectionObject Detection+1TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series
Time series forecasting plays a critical role in domains such as transportation, energy, and meteorology. Despite their success, modern deep forecasting models are typically trained to minimize point-wise prediction loss…
Time Series Forecasting