Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks
Is more data always better to train vision-and-language models? We study knowledge transferability in multi-modal tasks. The current tendency in machine learning is to assume that by joining multiple datasets from different tasks their overall performance will improve. However, we show that not all the knowledge transfers well or has a positive impact on related tasks, even when they share a common goal. We conduct an exhaustive analysis based on hundreds of cross-experiments on 12 vision-and-language tasks categorized in 4 groups. Whereas tasks in the same group are prone to improve each other, results show that this is not always the case. Other factors such as dataset size or pre-training stage have also a great impact on how well the knowledge is transferred.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Linguistic Knowledge and Transferability of Contextual Representations
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language. To shed light on…
Language ModelingLanguage ModellingUnderstanding Knowledge Transferability for Transfer Learning: A Survey
Transfer learning has become an essential paradigm in artificial intelligence, enabling the transfer of knowledge from a source task to improve performance on a target task. This approach, particularly through techniques…
Transfer LearningCLAP: Learning Transferable Binary Code Representations with Natural Language Supervision
Binary code representation learning has shown significant performance in binary analysis tasks. But existing solutions often have poor transferability, particularly in few-shot and zero-shot scenarios where few or no tra…
Representation LearningTransfer LearningLarge Language Models Are Cross-Lingual Knowledge-Free Reasoners
Large Language Models have demonstrated impressive reasoning capabilities across multiple languages. However, the relationship between capabilities in different languages is less explored. In this work, we decompose the …
Cross-Lingual TransferLogical ReasoningRetrievalBoosting the Adversarial Transferability of Surrogate Models with Dark Knowledge
Deep neural networks (DNNs) are vulnerable to adversarial examples. And, the adversarial examples have transferability, which means that an adversarial example for a DNN model can fool another model with a non-trivial pr…
Adversarial AttackFace VerificationImage Classification