Theoretical Guarantees of Transfer Learning
Transfer learning has been proven effective when within-target labeled data is scarce. A lot of works have developed successful algorithms and empirically observed positive transfer effect that improves target generalization error using source knowledge. However, theoretical analysis of transfer learning is more challenging due to the nature of the problem and thus is less studied. In this report, we do a survey of theoretical works in transfer learning and summarize key theoretical guarantees that prove the effectiveness of transfer learning. The theoretical bounds are derived using model complexity and learning algorithm stability. As we should see, these works exhibit a trade-off between tight bounds and restrictive assumptions. Moreover, we also prove a new generalization bound for the multi-source transfer learning problem using the VC-theory, which is more informative than the one proved in previous work.
Code (0)
등록된 구현이 없습니다.
Tasks
Transfer LearningSimilar Papers 제목 키워드 기반
Transfer Learning Under High-Dimensional Graph Convolutional Regression Model for Node Classification
Node classification is a fundamental task, but obtaining node classification labels can be challenging and expensive in many real-world scenarios. Transfer learning has emerged as a promising solution to address this cha…
ClassificationNode ClassificationregressionTransfer LearningPessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning
Transfer reinforcement learning aims to derive a near-optimal policy for a target environment with limited data by leveraging abundant data from related source domains. However, it faces two key challenges: the lack of p…
reinforcement-learningReinforcement LearningTransfer LearningTransfer Reinforcement LearningTransfer Learning of Graph Neural Networks with Ego-graph Information Maximization
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. Howev…
Knowledge GraphsTransfer LearningTransfer Learning for a Class of Cascade Dynamical Systems
This work considers the problem of transfer learning in the context of reinforcement learning. Specifically, we consider training a policy in a reduced order system and deploying it in the full state system. The motivati…
reinforcement-learningReinforcement LearningTransfer LearningGeneralisation Guarantees for Continual Learning with Orthogonal Gradient Descent
In Continual Learning settings, deep neural networks are prone to Catastrophic Forgetting. Orthogonal Gradient Descent was proposed to tackle the challenge. However, no theoretical guarantees have been proven yet. We pre…
Continual LearningTransfer Learning