Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier
Human learners have the natural ability to use knowledge gained in one setting for learning in a different but related setting. This ability to transfer knowledge from one task to another is essential for effective learning. In this paper, we study transfer learning in the context of nonparametric classification based on observations from different distributions under the posterior drift model, which is a general framework and arises in many practical problems. We first establish the minimax rate of convergence and construct a rate-optimal two-sample weighted $K$-NN classifier. The results characterize precisely the contribution of the observations from the source distribution to the classification task under the target distribution. A data-driven adaptive classifier is then proposed and is shown to simultaneously attain within a logarithmic factor of the optimal rate over a large collection of parameter spaces. Simulation studies and real data applications are carried out where the numerical results further illustrate the theoretical analysis. Extensions to the case of multiple source distributions are also considered.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationTransfer LearningSimilar Papers 제목 키워드 기반
Minimax And Adaptive Transfer Learning for Nonparametric Classification under Distributed Differential Privacy Constraints
This paper considers minimax and adaptive transfer learning for nonparametric classification under the posterior drift model with distributed differential privacy constraints. Our study is conducted within a heterogeneou…
ClassificationTransfer LearningTransfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure
Transfer learning for nonparametric regression is considered. We first study the non-asymptotic minimax risk for this problem and develop a novel estimator called the confidence thresholding estimator, which is shown to …
regressionTransfer LearningModel-Robust and Adaptive-Optimal Transfer Learning for Tackling Concept Shifts in Nonparametric Regression
When concept shifts and sample scarcity are present in the target domain of interest, nonparametric regression learners often struggle to generalize effectively. The technique of transfer learning remedies these issues b…
Transfer LearningAdaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression
We show both adaptive and non-adaptive minimax rates of convergence for a family of weighted Laplacian-Eigenmap based nonparametric regression methods, when the true regression function belongs to a Sobolev space and the…
regressionBenign overfitting and adaptive nonparametric regression
In the nonparametric regression setting, we construct an estimator which is a continuous function interpolating the data points with high probability, while attaining minimax optimal rates under mean squared risk on the …
regression