Lifelong Learning via Online Leverage Score Sampling
In order to mimic the human ability of continual acquisition and transfer of knowledge across various tasks, a learning system needs the capability for life-long learning, effectively utilizing the previously acquired skills. As such, the key challenge is to transfer and generalize the knowledge learned from one task to other tasks, avoiding interference from previous knowledge and improving the overall performance. In this paper, within the continual learning paradigm, we introduce a method that effectively forgets the less useful data samples continuously across different tasks. The method uses statistical leverage score information to measure the importance of the data samples in every task and adopts frequent directions approach to enable a life-long learning property. This effectively maintains a constant training size across all tasks. We first provide some mathematical intuition for the method and then demonstrate its effectiveness with experiments on variants of MNIST and CIFAR100 datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningLifelong learningSimilar Papers 제목 키워드 기반
Importance Sampling for Nonlinear Models
While norm-based and leverage-score-based methods have been extensively studied for identifying "important" data points in linear models, analogous tools for nonlinear models remain significantly underdeveloped. By intro…
Outlier DetectionUncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation
Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amoun…
2kMambaSequential RecommendationOptimal Sampling Designs for Multi-dimensional Streaming Time Series with Application to Power Grid Sensor Data
The Internet of Things (IoT) system generates massive high-speed temporally correlated streaming data and is often connected with online inference tasks under computational or energy constraints. Online analysis of these…
Computational EfficiencyTime SeriesTime Series AnalysisCLIP model is an Efficient Online Lifelong Learner
Online Lifelong Learning (OLL) addresses the challenge of learning from continuous and non-stationary data streams. Existing online lifelong learning methods based on image classification models often require preset cond…
image-classificationImage ClassificationLifelong learningmodel+1Embodied Lifelong Learning for Task and Motion Planning
A robot deployed in a home over long stretches of time faces a true lifelong learning problem. As it seeks to provide assistance to its users, the robot should leverage any accumulated experience to improve its own knowl…
Lifelong learningMotion PlanningTask and Motion Planning