Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
Non-Centralized Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices such as vehicles and servers to handle streaming data from a joint non-stationary environment. To achieve high reliability and scalability in deploying this paradigm in distributed systems, it is essential to conquer challenges stemming from both spatial and temporal dimensions, manifesting as distribution shifts, catastrophic forgetting, heterogeneity, and privacy issues. This survey focuses on a comprehensive examination of the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices. We begin with an introduction to the background and fundamentals of non-centralized learning and continual learning. Then, we review existing solutions from three levels to represent how existing techniques alleviate the catastrophic forgetting and distribution shift. Additionally, we delve into the various types of heterogeneity issues, security, and privacy attributes, as well as real-world applications across three prevalent scenarios. Furthermore, we establish a large-scale benchmark to revisit this problem and analyze the performance of the state-of-the-art NCCL approaches. Finally, we discuss the important challenges and future research directions in NCCL.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Federated Continual Learning via Knowledge Fusion: A Survey
Data privacy and silos are nontrivial and greatly challenging in many real-world applications. Federated learning is a decentralized approach to training models across multiple local clients without the exchange of raw d…
Continual LearningFederated LearningSurveyFederated and continual learning for classification tasks in a society of devices
Today we live in a context in which devices are increasingly interconnected and sensorized and are almost ubiquitous. Deep learning has become in recent years a popular way to extract knowledge from the huge amount of da…
Continual LearningFederated LearningGeneral ClassificationDesigning Object Detection Models for TinyML: Foundations, Comparative Analysis, Challenges, and Emerging Solutions
Object detection (OD) has become vital for numerous computer vision applications, but deploying it on resource-constrained IoT devices presents a significant challenge. These devices, often powered by energy-efficient mi…
Neural Architecture SearchKnowledge DistillationObject DetectionTowards Edge General Intelligence via Large Language Models: Opportunities and Challenges
Edge Intelligence (EI) has been instrumental in delivering real-time, localized services by leveraging the computational capabilities of edge networks. The integration of Large Language Models (LLMs) empowers EI to evolv…
SurveyDealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is mu…
Decision MakingDeep Reinforcement LearningMeta-LearningMulti-agent Reinforcement Learning+3