A Multi-Modal Federated Learning Framework for Remote Sensing Image Classification
Federated learning (FL) enables the collaborative training of deep neural networks across decentralized data archives (i.e., clients) without sharing the local data of the clients. Most of the existing FL methods assume that the data distributed across all clients is associated with the same data modality. However, remote sensing (RS) images present in different clients can be associated with diverse data modalities. The joint use of the multi-modal RS data can significantly enhance classification performance. To effectively exploit decentralized and unshared multi-modal RS data, our paper introduces a novel multi-modal FL framework for RS image classification problems. The proposed framework comprises three modules: 1) multi-modal fusion (MF); 2) feature whitening (FW); and 3) mutual information maximization (MIM). The MF module employs iterative model averaging to facilitate learning without accessing multi-modal training data on clients. The FW module aims to address the limitations of training data heterogeneity by aligning data distributions across clients. The MIM module aims to model mutual information by maximizing the similarity between images from different modalities. For the experimental analyses, we focus our attention on multi-label classification and pixel-based classification tasks in RS. The results obtained using two benchmark archives show the effectiveness of the proposed framework when compared to state-of-the-art algorithms in the literature. The code of the proposed framework will be available at https://git.tu-berlin.de/rsim/multi-modal-FL.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated Learningimage-classificationImage ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONRemote Sensing Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models
Remote sensing data is often distributed across multiple institutions, and due to privacy concerns and data-sharing restrictions, leveraging large-scale datasets in a centralized training framework is challenging. Federa…
Federated Learningimage-classificationImage ClassificationPrompt Learning+3FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clients
With the rapid development of imaging sensor technology in the field of remote sensing, multi-modal remote sensing data fusion has emerged as a crucial research direction for land cover classification tasks. While diffus…
DenoisingFederated Learningimage-classificationImage Classification+1Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning
The development of federated learning (FL) methods, which aim to learn from distributed databases (i.e., clients) without accessing data on clients, has recently attracted great attention. Most of these methods assume th…
Federated Learningimage-classificationImage ClassificationSAFE: Self-Adjustment Federated Learning Framework for Remote Sensing Collaborative Perception
The rapid increase in remote sensing satellites has led to the emergence of distributed space-based observation systems. However, existing distributed remote sensing models often rely on centralized training, resulting i…
Computational EfficiencyFederated Learningimage-classificationImage Classification+1Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentation
Remote sensing semantic segmentation (RSS) is an essential technology in earth observation missions. Due to concerns over geographic information security, data privacy, storage bottleneck and industry competition, high-q…
Earth ObservationFederated LearningPrivacy PreservingSemantic Segmentation