Learning More Universal Representations for Transfer-Learning
A representation is supposed universal if it encodes any element of the visual world (e.g., objects, scenes) in any configuration (e.g., scale, context). While not expecting pure universal representations, the goal in the literature is to improve the universality level, starting from a representation with a certain level. To do so, the state-of-the-art consists in learning CNN-based representations on a diversified training problem (e.g., ImageNet modified by adding annotated data). While it effectively increases universality, such approach still requires a large amount of efforts to satisfy the needs in annotated data. In this work, we propose two methods to improve universality, but pay special attention to limit the need of annotated data. We also propose a unified framework of the methods based on the diversifying of the training problem. Finally, to better match Atkinson's cognitive study about universal human representations, we proposed to rely on the transfer-learning scheme as well as a new metric to evaluate universality. This latter, aims us to demonstrates the interest of our methods on 10 target-problems, relating to the classification task and a variety of visual domains.
Code (1)
Tasks
Transfer LearningSimilar Papers 제목 키워드 기반
Cross-lingual Semantic Role Labeling with Model Transfer
Prior studies show that cross-lingual semantic role labeling (SRL) can be achieved by model transfer under the help of universal features. In this paper, we fill the gap of cross-lingual SRL by proposing an end-to-end SR…
Semantic Role LabelingUniversal Successor Representations for Transfer Reinforcement Learning
The objective of transfer reinforcement learning is to generalize from a set of previous tasks to unseen new tasks. In this work, we focus on the transfer scenario where the dynamics among tasks are the same, but their g…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Transfer Learning+1Towards Universal Sequence Representation Learning for Recommender Systems
In order to develop effective sequential recommenders, a series of sequence representation learning (SRL) methods are proposed to model historical user behaviors. Most existing SRL methods rely on explicit item IDs for d…
Mixture-of-ExpertsRecommendation SystemsRepresentation LearningSufiSent - Universal Sentence Representations Using Suffix Encodings
Computing universal distributed representations of sentences is a fundamental task in natural language processing. We propose a method to learn such representations by encoding the suffixes of word sequences in a sentenc…
Natural Language InferenceSentenceSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however no…
Cross-Lingual Natural Language InferenceNatural Language InferenceSemantic Textual SimilaritySentence+2