Transferability and Hardness of Supervised Classification Tasks
We propose a novel approach for estimating the difficulty and transferability of supervised classification tasks. Unlike previous work, our approach is solution agnostic and does not require or assume trained models. Instead, we estimate these values using an information theoretic approach: treating training labels as random variables and exploring their statistics. When transferring from a source to a target task, we consider the conditional entropy between two such variables (i.e., label assignments of the two tasks). We show analytically and empirically that this value is related to the loss of the transferred model. We further show how to use this value to estimate task hardness. We test our claims extensively on three large scale data sets -- CelebA (40 tasks), Animals with Attributes 2 (85 tasks), and Caltech-UCSD Birds 200 (312 tasks) -- together representing 437 classification tasks. We provide results showing that our hardness and transferability estimates are strongly correlated with empirical hardness and transferability. As a case study, we transfer a learned face recognition model to CelebA attribute classification tasks, showing state of the art accuracy for tasks estimated to be highly transferable.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeClassificationFace RecognitionGeneral ClassificationSimilar Papers 제목 키워드 기반
Improving the Transferability of Supervised Pretraining with an MLP Projector
The pretrain-finetune paradigm is a classical pipeline in visual learning. Recent progress on unsupervised pretraining methods showed superior transfer performance to their supervised counterparts. While a few works atte…
domain classificationRevisiting the Transferability of Supervised Pretraining: an MLP Perspective
The pretrain-finetune paradigm is a classical pipeline in visual learning. Recent progress on unsupervised pretraining methods shows superior transfer performance to their supervised counterparts. This paper revisits thi…
domain classificationimage-classificationImage ClassificationLinear evaluation+3Diverse Imagenet Models Transfer Better
A commonly accepted hypothesis is that models with higher accuracy on Imagenet perform better on other downstream tasks, leading to much research dedicated to optimizing Imagenet accuracy. Recently this hypothesis has be…
DiversityMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONDiscriminability-Transferability Trade-Off: An Information-Theoretic Perspective
This work simultaneously considers the discriminability and transferability properties of deep representations in the typical supervised learning task, i.e., image classification. By a comprehensive temporal analysis, we…
image-classificationImage ClassificationTransfer LearningDeeply Unsupervised Patch Re-Identification for Pre-training Object Detectors
Unsupervised pre-training aims at learning transferable features that are beneficial for downstream tasks. However, most state-of-the-art unsupervised methods concentrate on learning global representations for image-leve…
Objectobject-detectionObject DetectionRepresentation Learning+1