Learnings from Federated Learning in the Real world
Federated Learning (FL) applied to real world data may suffer from several idiosyncrasies. One such idiosyncrasy is the data distribution across devices. Data across devices could be distributed such that there are some "heavy devices" with large amounts of data while there are many "light users" with only a handful of data points. There also exists heterogeneity of data across devices. In this study, we evaluate the impact of such idiosyncrasies on Natural Language Understanding (NLU) models trained using FL. We conduct experiments on data obtained from a large scale NLU system serving thousands of devices and show that simple non-uniform device selection based on the number of interactions at each round of FL training boosts the performance of the model. This benefit is further amplified in continual FL on consecutive time periods, where non-uniform sampling manages to swiftly catch up with FL methods using all data at once.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningNatural Language UnderstandingSimilar Papers 제목 키워드 기반
Stochastic flexibility needs assessment: learnings from H2020 EUniversal's German demonstration
Operational flexibility needs assessment (FNA) is crucial for system operators to plan/procure flexible resources in order to avoid probable network issues. We implemented an FNA tool in the framework of the H2020 EUnive…
Comparative assessment of federated and centralized machine learning
Federated Learning (FL) is a privacy preserving machine learning scheme, where training happens with data federated across devices and not leaving them to sustain user privacy. This is ensured by making the untrained or …
BIG-bench Machine LearningFederated LearningPrivacy PreservingArchitecture Agnostic Federated Learning for Neural Networks
With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep neural network model in a FL setting prov…
Federated LearningFederated Transfer Learning with Multimodal Data
Smart cars, smartphones and other devices in the Internet of Things (IoT), which usually have more than one sensors, produce multimodal data. Federated Learning supports collecting a wealth of multimodal data from differ…
Federated LearningSelf-Supervised LearningTransfer LearningDynamically Addressing Unseen Rumor via Continual Learning
Rumors are often associated with newly emerging events, thus, an ability to deal with unseen rumors is crucial for a rumor veracity classification model. Previous works address this issue by improving the model's general…
Continual LearningVeracity Classification