Closed-loop Control for Online Continual Learning
Online class-incremental continual learning (CL) deals with the sequential task learning problem in a realistic non-stationary setting with a single-pass through of data. Replay-based CL methods have shown promising results in several online class-incremental continual learning benchmarks. However, these replay methods typically assume pre-defined and fixed replay dynamics, which is suboptimal. This paper introduces a closed-loop continual learning framework, which obtains a real-time feedback learning signal via an additional test memory and then adapts the replay dynamics accordingly. More specifically, we propose a reinforcement learning-based method to dynamically adjust replay hyperparameters online to balance the stability and plasticity trade-off in continual learning. To address the non-stationarity in the continual learning environment, we employ a Q function with task-specific and task-shared components to support fast adaptation. The proposed method is applied to improve state-of-the-art replay-based methods and achieves superior performance on popular benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Non-Conservative Data-driven Safe Control Design for Nonlinear Systems with Polyhedral Safe Sets
This paper presents a data-driven nonlinear safe control design approach for discrete-time systems under parametric uncertainties and additive disturbances. We first characterize a new control structure from which a data…
Computational EfficiencyStability-Preserving Online Adaptation of Neural Closed-loop Maps
The growing complexity of modern control tasks calls for controllers that can react online as objectives and disturbances change, while preserving closed-loop stability. Recent approaches for improving the performance of…
Data-Driven Robust Stabilization with RobustDomain of Attraction Estimate for Nonlinear Discrete-Time Systems
Nonlinear robust control is pursued by overcoming the drawback of linear robust control that it ignores available information about existing nonlinearities and the resulting controllers may be too conservative, especiall…
High-Precision and High-Efficiency Trajectory Tracking for Excavators Based on Closed-Loop Dynamics
The complex nonlinear dynamics of hydraulic excavators, such as time delays and control coupling, pose significant challenges to achieving high-precision trajectory tracking. Traditional control methods often fall short …
Continual LearningOn the Design of Safe Continual RL Methods for Control of Nonlinear Systems
Reinforcement learning (RL) algorithms have been successfully applied to control tasks associated with unmanned aerial vehicles and robotics. In recent years, safe RL has been proposed to allow the safe execution of RL a…
Continual LearningMuJoCoReinforcement Learning (RL)