Prototype-Based Layered Federated Cross-Modal Hashing

2022-10-27 · Jiale Liu, Yu-Wei Zhan, Xin Luo, Zhen-Duo Chen, Yongxin Wang, Xin-Shun Xu

Recently, deep cross-modal hashing has gained increasing attention. However, in many practical cases, data are distributed and cannot be collected due to privacy concerns, which greatly reduces the cross-modal hashing performance on each client. And due to the problems of statistical heterogeneity, model heterogeneity, and forcing each client to accept the same parameters, applying federated learning to cross-modal hash learning becomes very tricky. In this paper, we propose a novel method called prototype-based layered federated cross-modal hashing. Specifically, the prototype is introduced to learn the similarity between instances and classes on server, reducing the impact of statistical heterogeneity (non-IID) on different clients. And we monitor the distance between local and global prototypes to further improve the performance. To realize personalized federated learning, a hypernetwork is deployed on server to dynamically update different layers' weights of local model. Experimental results on benchmark datasets show that our method outperforms state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2210.15678

Code (0)

등록된 구현이 없습니다.

Tasks

Federated Learning Personalized Federated Learning

Methods 이 논문이 사용한 방법론

HyperNetwork A HyperNetwork is a network that generates weights for a main network. The behavior of the main network is the same with any usual neural network: it learns to map some raw…

FedHAP: Federated Hashing with Global Prototypes for Cross-silo Retrieval

2022-07-12 · Meilin Yang, Jian Xu, Yang Liu, Wenbo Ding

Deep hashing has been widely applied in large-scale data retrieval due to its superior retrieval efficiency and low storage cost. However, data are often scattered in data silos with privacy concerns, so performing centr…

Deep Hashing Federated Learning Retrieval

Cluster-wise Unsupervised Hashing for Cross-Modal Similarity Search

2019-11-11 · Lu Wang, Jie Yang

Large-scale cross-modal hashing similarity retrieval has attracted more and more attention in modern search applications such as search engines and autopilot, showing great superiority in computation and storage. However…

Clustering Retrieval

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

2026-07-07 · Aavash Chhetri, Bibek Niroula, Eduard Vazquez, Yash Raj Shrestha 외 arxiv

In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available…

Federated Learning

Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality

2024-01-25 · Huy Q. Le, Chu Myaet Thwal, Yu Qiao, Ye Lin Tun 외

Multimodal federated learning (MFL) has emerged as a decentralized machine learning paradigm, allowing multiple clients with different modalities to collaborate on training a machine learning model across diverse data so…

cross-modal alignment Federated Learning

Unsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashing

2026-06-30 · Runhao Li, Xiaoxu Ma, Zhenyu Weng, Yue Zhang 외 arxiv

Compared to supervised cross-modal hashing (CMH), unsupervised CMH reduces the reliance on manual labeling by learning binary codes from unlabeled image-text pairs. However, existing unsupervised CMH methods often rely o…

Cross-Modal Retrieval Contrastive Learning