Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

2021-06-09 · Bokun Wang, Zhuoning Yuan, Yiming Ying, Tianbao Yang

In recent years, model-agnostic meta-learning (MAML) has become a popular research area. However, the stochastic optimization of MAML is still underdeveloped. Existing MAML algorithms rely on the ``episode'' idea by sampling a few tasks and data points to update the meta-model at each iteration. Nonetheless, these algorithms either fail to guarantee convergence with a constant mini-batch size or require processing a large number of tasks at every iteration, which is unsuitable for continual learning or cross-device federated learning where only a small number of tasks are available per iteration or per round. To address these issues, this paper proposes memory-based stochastic algorithms for MAML that converge with vanishing error. The proposed algorithms require sampling a constant number of tasks and data samples per iteration, making them suitable for the continual learning scenario. Moreover, we introduce a communication-efficient memory-based MAML algorithm for personalized federated learning in cross-device (with client sampling) and cross-silo (without client sampling) settings. Our theoretical analysis improves the optimization theory for MAML, and our empirical results corroborate our theoretical findings. Interested readers can access our code at \url{https://github.com/bokun-wang/moml}.

📄 PDF Abstract BibTeX arXiv:2106.04911

Code (1)

bokun-wang/moml 공식 구현 pytorch

Tasks

Continual Learning Federated Learning Meta-Learning Personalized Federated Learning Stochastic Optimization

Methods 이 논문이 사용한 방법론

MAML 설명 없음

MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation

2020-07-07 · Manqing Dong, Feng Yuan, Lina Yao, Xiwei Xu 외

A common challenge for most current recommender systems is the cold-start problem. Due to the lack of user-item interactions, the fine-tuned recommender systems are unable to handle situations with new users or new items…

Meta-Learning Recommendation Systems

Personalized Adaptive Meta Learning for Cold-start User Preference Prediction

2020-12-22 · Runsheng Yu, Yu Gong, Xu He, Bo An 외

A common challenge in personalized user preference prediction is the cold-start problem. Due to the lack of user-item interactions, directly learning from the new users' log data causes serious over-fitting problem. Rece…

Few-Shot Learning Meta-Learning

EMO: Episodic Memory Optimization for Few-Shot Meta-Learning

2023-06-08 · Yingjun Du, Jiayi Shen, XianTong Zhen, Cees G. M. Snoek

Few-shot meta-learning presents a challenge for gradient descent optimization due to the limited number of training samples per task. To address this issue, we propose an episodic memory optimization for meta-learning, w…

Few-Shot Learning Meta-Learning

On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies

2022-09-21 · Haozhi Wang, Qing Wang, Yunfeng Shao, Dong Li 외

Modern meta-reinforcement learning (Meta-RL) methods are mainly developed based on model-agnostic meta-learning, which performs policy gradient steps across tasks to maximize policy performance. However, the gradient con…

continuous-control Continuous Control Meta-Learning Meta Reinforcement Learning+3

One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment

2026-01-26 · Hongru Cai, Yongqi Li, Tiezheng Yu, Fengbin Zhu 외 arxiv

Alignment of Large Language Models (LLMs) aims to align outputs with human preferences, and personalized alignment further adapts models to individual users. This relies on personalized reward models that capture user-sp…