LEEP: A New Measure to Evaluate Transferability of Learned Representations
We introduce a new measure to evaluate the transferability of representations learned by classifiers. Our measure, the Log Expected Empirical Prediction (LEEP), is simple and easy to compute: when given a classifier trained on a source data set, it only requires running the target data set through this classifier once. We analyze the properties of LEEP theoretically and demonstrate its effectiveness empirically. Our analysis shows that LEEP can predict the performance and convergence speed of both transfer and meta-transfer learning methods, even for small or imbalanced data. Moreover, LEEP outperforms recently proposed transferability measures such as negative conditional entropy and H scores. Notably, when transferring from ImageNet to CIFAR100, LEEP can achieve up to 30% improvement compared to the best competing method in terms of the correlations with actual transfer accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
TransferabilityTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks
With the increasing abundance of pretrained models in recent years, the problem of selecting the best pretrained checkpoint for a particular downstream classification task has been gaining increased attention. Although s…
Learning TheoryModel SelectionTransferabilityTransfer LearningSleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals
Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We curate a large polysomnography dataset from over 14,000 participants compr…
Contrastive LearningRepresentation LearningTransferability of Representations Learned using Supervised Contrastive Learning Trained on a Multi-Domain Dataset
Contrastive learning has shown to learn better quality representations than models trained using cross-entropy loss. They also transfer better to downstream datasets from different domains. However, little work has been …
Contrastive LearningLinear evaluationLearning cortical representations through perturbed and adversarial dreaming
Humans and other animals learn to extract general concepts from sensory experience without extensive teaching. This ability is thought to be facilitated by offline states like sleep where previous experiences are systemi…
Learning Semantic RepresentationsA Deep Learning Approach with an Attention Mechanism for Automatic Sleep Stage Classification
Automatic sleep staging is a challenging problem and state-of-the-art algorithms have not yet reached satisfactory performance to be used instead of manual scoring by a sleep technician. Much research has been done to fi…
Automatic Sleep Stage ClassificationDimensionality ReductionGeneral ClassificationSleep Staging