Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality
This paper considers the estimation and prediction of a high-dimensional linear regression in the setting of transfer learning, using samples from the target model as well as auxiliary samples from different but possibly related regression models. When the set of "informative" auxiliary samples is known, an estimator and a predictor are proposed and their optimality is established. The optimal rates of convergence for prediction and estimation are faster than the corresponding rates without using the auxiliary samples. This implies that knowledge from the informative auxiliary samples can be transferred to improve the learning performance of the target problem. In the case that the set of informative auxiliary samples is unknown, we propose a data-driven procedure for transfer learning, called Trans-Lasso, and reveal its robustness to non-informative auxiliary samples and its efficiency in knowledge transfer. The proposed procedures are demonstrated in numerical studies and are applied to a dataset concerning the associations among gene expressions. It is shown that Trans-Lasso leads to improved performance in gene expression prediction in a target tissue by incorporating the data from multiple different tissues as auxiliary samples.
Code (1)
Tasks
PredictionregressionTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On Hypothesis Transfer Learning of Functional Linear Models
We study the transfer learning (TL) for the functional linear regression (FLR) under the Reproducing Kernel Hilbert Space (RKHS) framework, observing the TL techniques in existing high-dimensional linear regression is no…
regressionTransfer LearningNonlinear Prediction of Multidimensional Signals via Deep Regression with Applications to Image Coding
Deep convolutional neural networks (DCNN) have enjoyed great successes in many signal processing applications because they can learn complex, non-linear causal relationships from input to output. In this light, DCNNs are…
PredictionregressionSharp Convergence Rates for Forward Regression in High-Dimensional Sparse Linear Models
Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown r…
Model SelectionregressionVocal Bursts Intensity PredictionMinimum-Norm Interpolation Under Covariate Shift
Transfer learning is a critical part of real-world machine learning deployments and has been extensively studied in experimental works with overparameterized neural networks. However, even in the simplest setting of line…
regressionTransfer LearningHigh-dimensional Asymptotics of Generalization Performance in Continual Ridge Regression
Continual learning is motivated by the need to adapt to real-world dynamics in tasks and data distribution while mitigating catastrophic forgetting. Despite significant advances in continual learning techniques, the theo…
Continual Learning