On the Transferability of Representations in Neural Networks Between Datasets and Tasks
Deep networks, composed of multiple layers of hierarchical distributed representations, tend to learn low-level features in initial layers and transition to high-level features towards final layers. Paradigms such as transfer learning, multi-task learning, and continual learning leverage this notion of generic hierarchical distributed representations to share knowledge across datasets and tasks. Herein, we study the layer-wise transferability of representations in deep networks across a few datasets and tasks and note some interesting empirical observations.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningMulti-Task LearningTransfer LearningSimilar Papers 제목 키워드 기반
Task2Box: Box Embeddings for Modeling Asymmetric Task Relationships
Modeling and visualizing relationships between tasks or datasets is an important step towards solving various meta-tasks such as dataset discovery, multi-tasking, and transfer learning. However, many relationships, such …
Attributeimage-classificationImage ClassificationTransfer LearningTowards Understanding the Transferability of Deep Representations
Deep neural networks trained on a wide range of datasets demonstrate impressive transferability. Deep features appear general in that they are applicable to many datasets and tasks. Such property is in prevalent use in r…
Balancing Transferability and Discriminability for Unsupervised Domain Adaptation.
Unsupervised domain adaptation (UDA) aims to leverage a sufficiently labeled source domain to classify or represent the fully unlabeled target domain with a different distribution. Generally, the existing approaches try …
Domain AdaptationRepresentation LearningUnsupervised Domain AdaptationAdversarial Training Reduces Information and Improves Transferability
Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it…
Fast and Accurate Transferability Measurement for Heterogeneous Multivariate Data
Given a set of heterogeneous source datasets with their classifiers, how can we quickly find the most useful source dataset for a specific target task? We address the problem of measuring transferability between source a…
Decoder