Deep Class Incremental Learning from Decentralized Data

2022-03-11 · Xiaohan Zhang, Songlin Dong, Jinjie Chen, Qi Tian, Yihong Gong, Xiaopeng Hong

In this paper, we focus on a new and challenging decentralized machine learning paradigm in which there are continuous inflows of data to be addressed and the data are stored in multiple repositories. We initiate the study of data decentralized class-incremental learning (DCIL) by making the following contributions. Firstly, we formulate the DCIL problem and develop the experimental protocol. Secondly, we introduce a paradigm to create a basic decentralized counterpart of typical (centralized) class-incremental learning approaches, and as a result, establish a benchmark for the DCIL study. Thirdly, we further propose a Decentralized Composite knowledge Incremental Distillation framework (DCID) to transfer knowledge from historical models and multiple local sites to the general model continually. DCID consists of three main components namely local class-incremental learning, collaborated knowledge distillation among local models, and aggregated knowledge distillation from local models to the general one. We comprehensively investigate our DCID framework by using different implementations of the three components. Extensive experimental results demonstrate the effectiveness of our DCID framework. The codes of the baseline methods and the proposed DCIL will be released at https://github.com/zxxxxh/DCIL.

📄 PDF Abstract BibTeX arXiv:2203.05984

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Incremental Learning Knowledge Distillation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning

2022-02-07 · Hao Chen, Yu Ye, Ming Xiao, Mikael Skoglund

Machine learning (ML) is a key technique for big-data-driven modelling and analysis of massive Internet of Things (IoT) based intelligent and ubiquitous computing. For fast-increasing applications and data amounts, distr…

BIG-bench Machine Learning

PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning

2024-01-04 · Haiyang Guo, Fei Zhu, Wenzhuo LIU, Xu-Yao Zhang 외

Existing federated learning methods have effectively dealt with decentralized learning in scenarios involving data privacy and non-IID data. However, in real-world situations, each client dynamically learns new classes, …

class-incremental learning Class Incremental Learning Federated Learning Incremental Learning

Few-Shot Class-Incremental Learning with Non-IID Decentralized Data

2024-09-18 · Cuiwei Liu, Siang Xu, Huaijun Qiu, Jing Zhang 외

Few-shot class-incremental learning is crucial for developing scalable and adaptive intelligent systems, as it enables models to acquire new classes with minimal annotated data while safeguarding the previously accumulat…

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Few-Shot Learning+2

CoMIX: A Multi-agent Reinforcement Learning Training Architecture for Efficient Decentralized Coordination and Independent Decision-Making

2023-08-21 · Giovanni Minelli, Mirco Musolesi

Robust coordination skills enable agents to operate cohesively in shared environments, together towards a common goal and, ideally, individually without hindering each other's progress. To this end, this paper presents C…

Decision Making Multi-agent Reinforcement Learning

Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents

2018-02-23 · ICML 2018 7 · Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang 외

We consider the problem of \emph{fully decentralized} multi-agent reinforcement learning (MARL), where the agents are located at the nodes of a time-varying communication network. Specifically, we assume that the reward …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)