CAML: Fast Context Adaptation via Meta-Learning
We propose CAML, a meta-learning method for fast adaptation that partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks. At test time, the context parameters are updated with one or several gradient steps on a task-specific loss that is backpropagated through the shared part of the network. Compared to approaches that adjust all parameters on a new task (e.g., MAML), our method can be scaled up to larger networks without overfitting on a single task, is easier to implement, and saves memory writes during training and network communication at test time for distributed machine learning systems. We show empirically that this approach outperforms MAML, is less sensitive to the task-specific learning rate, can capture meaningful task embeddings with the context parameters, and outperforms alternative partitionings of the parameter vectors.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningSimilar Papers 제목 키워드 기반
Contrastive Knowledge-Augmented Meta-Learning for Few-Shot Classification
Model agnostic meta-learning algorithms aim to infer priors from several observed tasks that can then be used to adapt to a new task with few examples. Given the inherent diversity of tasks arising in existing benchmarks…
ClassificationDiversityFew-Shot LearningMeta-LearningCumulative Meta-Learning from Active Learning Queries for Robustness to Spurious Correlations
Spurious correlations in real-world datasets cause machine learning models to rely on irrelevant patterns, undermining reliability, generalization, and fairness. Active learning offers a promising way to address this fai…
Active LearningThe Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in LLMs with Camlang
Large Language Models (LLMs) achieve gold-medal performance across many benchmarks, yet it remains unclear whether such success reflects genuine reasoning or pattern matching. From a cognitive science perspective, an inf…
Scalable Meta-Learning with Gaussian Processes
Meta-learning is a powerful approach that exploits historical data to quickly solve new tasks from the same distribution. In the low-data regime, methods based on the closed-form posterior of Gaussian processes (GP) toge…
Bayesian OptimizationGaussian ProcessesMeta-LearningConnection-Adaptive Meta-Learning
Meta-learning enables models to adapt to new environments rapidly with a few training examples. Current gradient-based meta-learning methods concentrate on finding good initialization (meta-weights) for learners, but ign…
Meta-Learning