A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Meta learning is a promising paradigm to enable skill transfer across tasks. Most previous methods employ the empirical risk minimization principle in optimization. However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of expected tail risk. We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningSimilar Papers 제목 키워드 기반
Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation
Meta-learning is a practical learning paradigm to transfer skills across tasks from a few examples. Nevertheless, the existence of task distribution shifts tends to weaken meta-learners' generalization capability, partic…
Meta-LearningA Simple Meta-learning Paradigm for Zero-shot Intent Classification with Mixture Attention Mechanism
Zero-shot intent classification is a vital and challenging task in dialogue systems, which aims to deal with numerous fast-emerging unacquainted intents without annotated training data. To obtain more satisfactory perfor…
Classificationintent-classificationIntent ClassificationMeta-Learning+3Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks
Graph neural networks have been demonstrated as a powerful paradigm for effectively learning graph-structured data on the web and mining content from it.Current leading graph models require a large number of labeled samp…
Few-Shot LearningGraph LearningMeta-LearningImproving the Robustness of Dense Retrievers Against Typos via Multi-Positive Contrastive Learning
Dense retrieval has become the new paradigm in passage retrieval. Despite its effectiveness on typo-free queries, it is not robust when dealing with queries that contain typos. Current works on improving the typo-robustn…
Contrastive LearningData AugmentationPassage RetrievalRetrievalRobustifying Sequential Neural Processes
When tasks change over time, meta-transfer learning seeks to improve the efficiency of learning a new task via both meta-learning and transfer-learning. While the standard attention has been effective in a variety of set…
Meta-LearningTransfer Learning