FDRMFL: Multimodal Federated Feature Extraction Model Based on Information Maximization and Contrastive Learning
We propose FDRMFL, a task-driven multimodal feature extraction framework for federated regression under non-IID data distributions. Extracting predictive features from high-dimensional multimodal inputs is particularly challenging in this setting: data cannot leave each client, local samples are scarce and heterogeneously distributed, and unsupervised dimensionality reduction discards task-relevant information while federated training introduces representation drift across communication rounds. FDRMFL addresses these challenges through a unified four-term local objective: MSE prediction loss, a correlation-based mutual information surrogate that preserves dependence between the fused representation and the continuous target, a symmetric KL penalty that aligns cross-modal latent distributions before fusion, and an InfoNCE-style contrastive loss that anchors local representations to the global consensus. Experiments on three synthetic and two real-world near-infrared spectroscopy datasets under non-IID federated partitions, with comprehensive ablation and sensitivity analyses, demonstrate that each component contributes to the framework's effectiveness. FDRMFL reduces mean MSE by 33.8% relative to the best traditional baseline (PCA) and by 43.0% relative to VAE in simulation, and attains the lowest overall mean MSE among six federated algorithms including FedAvg, FedProx, MOON, SCAFFOLD, and FedBN.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionContrastive LearningSimilar Papers 제목 키워드 기반
FedMultimodal: A Benchmark For Multimodal Federated Learning
Over the past few years, Federated Learning (FL) has become an emerging machine learning technique to tackle data privacy challenges through collaborative training. In the Federated Learning algorithm, the clients submit…
Emotion RecognitionFederated LearningMissing LabelsPilot: Building the Federated Multimodal Instruction Tuning Framework
In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices.…
General KnowledgeM$^{3}$D: A Multimodal, Multilingual and Multitask Dataset for Grounded Document-level Information Extraction
Multimodal information extraction (IE) tasks have attracted increasing attention because many studies have shown that multimodal information benefits text information extraction. However, existing multimodal IE datasets …
Relation ExtractionVisual GroundingFedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology
The fusion of complementary multimodal information is crucial in computational pathology for accurate diagnostics. However, existing multimodal learning approaches necessitate access to users' raw data, posing substantia…
Federated LearningPrivacy PreservingBuilding a Multimodal Dataset of Academic Paper for Keyword Extraction
Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks …
Keyword Extraction