Trace norm regularization for multi-task learning with scarce data

2022-02-14 · Etienne Boursier, Mikhail Konobeev, Nicolas Flammarion

Multi-task learning leverages structural similarities between multiple tasks to learn despite very few samples. Motivated by the recent success of neural networks applied to data-scarce tasks, we consider a linear low-dimensional shared representation model. Despite an extensive literature, existing theoretical results either guarantee weak estimation rates or require a large number of samples per task. This work provides the first estimation error bound for the trace norm regularized estimator when the number of samples per task is small. The advantages of trace norm regularization for learning data-scarce tasks extend to meta-learning and are confirmed empirically on synthetic datasets.

📄 PDF Abstract BibTeX arXiv:2202.06742

Code (1)

michaelkonobeev/meta 공식 구현 jax

Tasks

Meta-Learning Multi-Task Learning

Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm

2010-12-01 · NeurIPS 2010 12 · Nathan Srebro, Ruslan R. Salakhutdinov

We show that matrix completion with trace-norm regularization can be significantly hurt when entries of the matrix are sampled non-uniformly, but that a properly weighted version of the trace-norm regularizer works well …

Collaborative Filtering Matrix Completion

Proximal Riemannian Pursuit for Large-Scale Trace-Norm Minimization

2016-06-01 · CVPR 2016 6 · Mingkui Tan, Shijie Xiao, Junbin Gao, Dong Xu 외

Trace-norm regularization plays an important role in many areas such as machine learning and computer vision. Solving trace-norm regularized Trace-norm regularization plays an important role in many areas such as compute…

BIG-bench Machine Learning Clustering Matrix Completion

Spectral k-Support Norm Regularization

2014-12-01 · NeurIPS 2014 12 · Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos

The $k$-support norm has successfully been applied to sparse vector prediction problems. We observe that it belongs to a wider class of norms, which we call the box-norms. Within this framework we derive an efficient alg…

Matrix Completion

Spectral Regularization: an Inductive Bias for Sequence Modeling

2022-11-04 · Kaiwen Hou, Guillaume Rabusseau

Various forms of regularization in learning tasks strive for different notions of simplicity. This paper presents a spectral regularization technique, which attaches a unique inductive bias to sequence modeling based on …

Inductive Bias

A New Convex Relaxation for Tensor Completion

2013-07-17 · NeurIPS 2013 12 · Bernardino Romera-Paredes, Massimiliano Pontil

We study the problem of learning a tensor from a set of linear measurements. A prominent methodology for this problem is based on a generalization of trace norm regularization, which has been used extensively for learnin…