Toward Improved Generalization: Meta Transfer of Self-supervised Knowledge on Graphs
Despite the remarkable success achieved by graph convolutional networks for functional brain activity analysis, the heterogeneity of functional patterns and the scarcity of imaging data still pose challenges in many tasks. Transferring knowledge from a source domain with abundant training data to a target domain is effective for improving representation learning on scarce training data. However, traditional transfer learning methods often fail to generalize the pre-trained knowledge to the target task due to domain discrepancy. Self-supervised learning on graphs can increase the generalizability of graph features since self-supervision concentrates on inherent graph properties that are not limited to a particular supervised task. We propose a novel knowledge transfer strategy by integrating meta-learning with self-supervised learning to deal with the heterogeneity and scarcity of fMRI data. Specifically, we perform a self-supervised task on the source domain and apply meta-learning, which strongly improves the generalizability of the model using the bi-level optimization, to transfer the self-supervised knowledge to the target domain. Through experiments on a neurological disorder classification task, we demonstrate that the proposed strategy significantly improves target task performance by increasing the generalizability and transferability of graph-based knowledge.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningRepresentation LearningSelf-Supervised LearningTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Meta-Learning via In-Context Learning
Unsupervised meta-learning aims to learn feature representations from unsupervised datasets that can transfer to downstream tasks with limited labeled data. In this paper, we propose a novel approach to unsupervised meta…
In-Context LearningMemorizationMeta-LearningSelf-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks
Self-supervised pre-training of transformer models has revolutionized NLP applications. Such pre-training with language modeling objectives provides a useful initial point for parameters that generalize well to new tasks…
General ClassificationLanguage ModelingLanguage ModellingMeta-Learning+1A Brief Summary of Interactions Between Meta-Learning and Self-Supervised Learning
This paper briefly reviews the connections between meta-learning and self-supervised learning. Meta-learning can be applied to improve model generalization capability and to construct general AI algorithms. Self-supervis…
Data AugmentationMeta-LearningSelf-Supervised LearningUnsupervised Pre-trainingBoosting Supervision with Self-Supervision for Few-shot Learning
We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervise…
Few-Shot LearningSelf-Supervised LearningSelf-Supervised Prototypical Transfer Learning for Few-Shot Classification
Most approaches in few-shot learning rely on costly annotated data related to the goal task domain during (pre-)training. Recently, unsupervised meta-learning methods have exchanged the annotation requirement for a reduc…
ClassificationFew-Shot LearningGeneral ClassificationMeta-Learning+4