Empirical Studies of Institutional Federated Learning For Natural Language Processing
Federated learning has sparkled new interests in the deep learning society to make use of isolated data sources from independent institutes. With the development of novel training tools, we have successfully deployed federated natural language processing networks on GPU-enabled server clusters. This paper demonstrates federated training of a popular NLP model, TextCNN, with applications in sentence intent classification. Furthermore, differential privacy is introduced to protect participants in the training process, in a manageable manner. Distinguished from previous client-level privacy protection schemes, the proposed differentially private federated learning procedure is defined in the dataset sample level, inherent with the applications among institutions instead of individual users. Optimal settings of hyper-parameters for the federated TextCNN model are studied through comprehensive experiments. We also evaluated the performance of federated TextCNN model under imbalanced data load configuration. Experiments show that, the sampling ratio has a large impact on the performance of the FL models, causing up to 38.4{\%} decrease in the test accuracy, while they are robust to different noise multiplier levels, with less than 3{\%} variance in the test accuracy. It is also found that the FL models are sensitive to data load balancedness among client datasets. When the data load is imbalanced, model performance dropped by up to 10{\%}.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningGPUintent-classificationIntent ClassificationSentenceSimilar Papers 제목 키워드 기반
Federated learning for violence incident prediction in a simulated cross-institutional psychiatric setting
Inpatient violence is a common and severe problem within psychiatry. Knowing who might become violent can influence staffing levels and mitigate severity. Predictive machine learning models can assess each patient's like…
Federated LearningFederated Learning for Privacy-Preserving Medical AI
This dissertation investigates privacy-preserving federated learning for Alzheimer's disease classification using three-dimensional MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI). Existing methodolo…
Federated LearningRepresentation Learning to Advance Multi-institutional Studies with Electronic Health Record Data
The adoption of EHRs has expanded opportunities to leverage data-driven algorithms in clinical care and research. A major bottleneck in effectively conducting multi-institutional EHR studies is the data heterogeneity acr…
Federated LearningGraph AttentionKnowledge GraphsRepresentation LearningFeTTL: Federated Template and Task Learning for Multi-Institutional Medical Imaging
Federated learning enables collaborative model training across geographically distributed medical centers while preserving data privacy. However, domain shifts and heterogeneity in data often lead to a degradation in mod…
Federated LearningMammoFL: Mammographic Breast Density Estimation using Federated Learning
In this study, we automate quantitative mammographic breast density estimation with neural networks and show that this tool is a strong use case for federated learning on multi-institutional datasets. Our dataset include…
Density EstimationFederated Learning