Coresets via Bilevel Optimization for Continual Learning and Streaming
Coresets are small data summaries that are sufficient for model training. They can be maintained online, enabling efficient handling of large data streams under resource constraints. However, existing constructions are limited to simple models such as k-means and logistic regression. In this work, we propose a novel coreset construction via cardinality-constrained bilevel optimization. We show how our framework can efficiently generate coresets for deep neural networks, and demonstrate its empirical benefits in continual learning and in streaming settings.
Code (1)
Tasks
Bilevel OptimizationContinual LearningregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bilevel Continual Learning
Continual learning (CL) studies the problem of learning a sequence of tasks, one at a time, such that the learning of each new task does not lead to the deterioration in performance on the previously seen ones while expl…
Bilevel OptimizationContinual LearningMeta-LearningProbabilistic Bilevel Coreset Selection
The goal of coreset selection in supervised learning is to produce a weighted subset of data, so that training only on the subset achieves similar performance as training on the entire dataset. Existing methods achieved …
Bilevel OptimizationContinual LearningBilevel Continual Learning
Continual learning aims to learn continuously from a stream of tasks and data in an online-learning fashion, being capable of exploiting what was learned previously to improve current and future tasks while still being a…
Bilevel OptimizationContinual LearningManagementTransfer LearningBilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm
Coreset is a small set that provides a data summary for a large dataset, such that training solely on the small set achieves competitive performance compared with a large dataset. In rehearsal-based continual learning, t…
Non-Convex Bilevel Optimization with Time-Varying Objective Functions
Bilevel optimization has become a powerful tool in a wide variety of machine learning problems. However, the current nonconvex bilevel optimization considers an offline dataset and static functions, which may not work we…
Bilevel Optimization