When More Data Doesn't Help: Limits of Adaptation in Multitask Learning
Multitask learning and related frameworks have achieved tremendous success in modern applications. In multitask learning problem, we are given a set of heterogeneous datasets collected from related source tasks and hope to enhance the performance above what we could hope to achieve by solving each of them individually. The recent work of arXiv:2006.15785 has showed that, without access to distributional information, no algorithm based on aggregating samples alone can guarantee optimal risk as long as the sample size per task is bounded. In this paper, we focus on understanding the statistical limits of multitask learning. We go beyond the no-free-lunch theorem in arXiv:2006.15785 by establishing a stronger impossibility result of adaptation that holds for arbitrarily large sample size per task. This improvement conveys an important message that the hardness of multitask learning cannot be overcame by having abundant data per task. We also discuss the notion of optimal adaptivity that may be of future interests.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Short Answer Grading: When Sorting Helps and When it Doesn't
Batch Normalized Recurrent Neural Networks
Recurrent Neural Networks (RNNs) are powerful models for sequential data that have the potential to learn long-term dependencies. However, they are computationally expensive to train and difficult to parallelize. Recent …
Language Modellingspeech-recognitionSpeech RecognitionSize doesn't matter: predicting physico- or biochemical properties based on dozens of molecules
The use of machine learning in chemistry has become a common practice. At the same time, despite the success of modern machine learning methods, the lack of data limits their use. Using a transfer learning methodology ca…
BIG-bench Machine LearningTransfer LearningUsing ML filters to help automated vulnerability repairs: when it helps and when it doesn't
[Context:] The acceptance of candidate patches in automated program repair has been typically based on testing oracles. Testing requires typically a costly process of building the application while ML models can be used …
Program RepairVulnerability DetectionTwo-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions
The use of language-model-based question-answering systems to aid humans in completing difficult tasks is limited, in part, by the unreliability of the text these systems generate. Using hard multiple-choice reading comp…
Language ModelingLanguage ModellingMultiple-choiceQuestion Answering+2