paper-with-me

Papers

FDRMFL: Multimodal Federated Feature Extraction Model Based on Information Maximization and Contrastive Learning

2025-11-30 · Haozhe Wu arxiv

We propose FDRMFL, a task-driven multimodal feature extraction framework for federated regression under non-IID data distributions. Extracting predictive features from high-dimensional multimodal inputs is particularly challenging in this setting: data cannot leave each client, local samples are scarce and heterogeneously distributed, and unsupervised dimensionality reduction discards task-relevant information while federated training introduces representation drift across communication rounds. FDRMFL addresses these challenges through a unified four-term local objective: MSE prediction loss, a correlation-based mutual information surrogate that preserves dependence between the fused representation and the continuous target, a symmetric KL penalty that aligns cross-modal latent distributions before fusion, and an InfoNCE-style contrastive loss that anchors local representations to the global consensus. Experiments on three synthetic and two real-world near-infrared spectroscopy datasets under non-IID federated partitions, with comprehensive ablation and sensitivity analyses, demonstrate that each component contributes to the framework's effectiveness. FDRMFL reduces mean MSE by 33.8% relative to the best traditional baseline (PCA) and by 43.0% relative to VAE in simulation, and attains the lowest overall mean MSE among six federated algorithms including FedAvg, FedProx, MOON, SCAFFOLD, and FedBN.

📄 PDF Abstract BibTeX arXiv:2512.02076

Code (0)

등록된 구현이 없습니다.

Tasks

Dimensionality ReductionContrastive Learning

Similar Papers 제목 키워드 기반

FedMultimodal: A Benchmark For Multimodal Federated Learning

2023-06-15 · Tiantian Feng, Digbalay Bose, Tuo Zhang, Rajat Hebbar 외

Over the past few years, Federated Learning (FL) has become an emerging machine learning technique to tackle data privacy challenges through collaborative training. In the Federated Learning algorithm, the clients submit…

Emotion RecognitionFederated LearningMissing Labels

Pilot: Building the Federated Multimodal Instruction Tuning Framework

2025-01-23 · Baochen Xiong, Xiaoshan Yang, Yaguang Song, YaoWei Wang 외

In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices.…

General Knowledge

M$^{3}$D: A Multimodal, Multilingual and Multitask Dataset for Grounded Document-level Information Extraction

2024-12-05 · Jiang Liu, Bobo Li, Xinran Yang, Na Yang 외

Multimodal information extraction (IE) tasks have attracted increasing attention because many studies have shown that multimodal information benefits text information extraction. However, existing multimodal IE datasets …

Relation ExtractionVisual Grounding

FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology

2024-02-24 · Yuanzhe Peng, Jieming Bian, Jie Xu

The fusion of complementary multimodal information is crucial in computational pathology for accurate diagnostics. However, existing multimodal learning approaches necessitate access to users' raw data, posing substantia…

Federated LearningPrivacy Preserving

Building a Multimodal Dataset of Academic Paper for Keyword Extraction

2026-06-30 · Jingyu Zhang, Xinyi Yan, Yi Xiang, Yingyi Zhang 외 arxiv

Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks …

Keyword Extraction