Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition
Federated Learning (FL) is a promising research paradigm that enables the collaborative training of machine learning models among various parties without the need for sensitive information exchange. Nonetheless, retaining data in individual clients introduces fundamental challenges to achieving performance on par with centrally trained models. Our study provides an extensive review of federated learning applied to visual recognition. It underscores the critical role of thoughtful architectural design choices in achieving optimal performance, a factor often neglected in the FL literature. Many existing FL solutions are tested on shallow or simple networks, which may not accurately reflect real-world applications. This practice restricts the transferability of research findings to large-scale visual recognition models. Through an in-depth analysis of diverse cutting-edge architectures such as convolutional neural networks, transformers, and MLP-mixers, we experimentally demonstrate that architectural choices can substantially enhance FL systems' performance, particularly when handling heterogeneous data. We study visual recognition models from five different architectural families on four challenging FL datasets. We also re-investigate the inferior performance convolution-based architectures in the FL setting and analyze the influence of normalization layers on the FL performance. Our findings emphasize the importance of architectural design for computer vision tasks in practical scenarios, effectively narrowing the performance gap between federated and centralized learning.Submission Number: 13840
Code (1)
Similar Papers 제목 키워드 기반
FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data leakage. While recent studies posit that …
Federated LearningAn Experimental Study of Data Heterogeneity in Federated Learning Methods for Medical Imaging
Federated learning enables multiple institutions to collaboratively train machine learning models on their local data in a privacy-preserving way. However, its distributed nature often leads to significant heterogeneity …
Federated LearningPrivacy PreservingArchitectural Blueprint For Heterogeneity-Resilient Federated Learning
This paper proposes a novel three tier architecture for federated learning to optimize edge computing environments. The proposed architecture addresses the challenges associated with client data heterogeneity and computa…
Edge-computingFederated LearningPrivacy PreservingBenchmarking Federated Learning for Semantic Datasets: Federated Scene Graph Generation
Federated learning (FL) has recently garnered attention as a data-decentralized training framework that enables the learning of deep models from locally distributed samples while keeping data privacy. Built upon the fram…
BenchmarkingFederated LearningGraph GenerationPanoptic Scene Graph Generation+1HPN: Personalized Federated Hyperparameter Optimization
Numerous research studies in the field of federated learning (FL) have attempted to use personalization to address the heterogeneity among clients, one of FL's most crucial and challenging problems. However, existing wor…
Federated LearningHyperparameter Optimization