Sparse coding for multitask and transfer learning
We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This assumption, together with the large quantity of available data in the multitask and transfer learning settings, allows a principled choice of the dictionary. We provide bounds on the generalization error of this approach, for both settings. Numerical experiments on one synthetic and two real datasets show the advantage of our method over single task learning, a previous method based on orthogonal and dense representation of the tasks and a related method learning task grouping.
Code (0)
등록된 구현이 없습니다.
Tasks
Dictionary LearningTransfer LearningSimilar Papers 제목 키워드 기반
Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms
Brain networks in fMRI are typically identified using spatial independent component analysis (ICA), yet mathematical constraints such as sparse coding and positivity both provide alternate biologically-plausible framewor…
Time Series AnalysisDiSparse: Disentangled Sparsification for Multitask Model Compression
Despite the popularity of Model Compression and Multitask Learning, how to effectively compress a multitask model has been less thoroughly analyzed due to the challenging entanglement of tasks in the parameter space. In …
modelModel CompressionAn Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems
Multitask learning assumes that models capable of learning from multiple tasks can achieve better quality and efficiency via knowledge transfer, a key feature of human learning. Though, state of the art ML models rely on…
Continual LearningFine-Grained Image Classificationimage-classificationImage Classification+1Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning
The increasing complexity of tasks in robotics demands efficient strategies for multitask and continual learning. Traditional models typically rely on a universal policy for all tasks, facing challenges such as high comp…
Continual LearningMixture-of-ExpertsSparse Multitask Learning for Efficient Neural Representation of Motor Imagery and Execution
In the quest for efficient neural network models for neural data interpretation and user intent classification in brain-computer interfaces (BCIs), learning meaningful sparse representations of the underlying neural subs…
Efficient Neural Networkintent-classificationIntent ClassificationMotor Imagery