FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition
Typical machine learning approaches require centralized data for model training, which may not be possible where restrictions on data sharing are in place due to, for instance, privacy and gradient protection. The recently proposed Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. However, we show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when the Federated Averaging (FedAvg) strategy is used due to the weight divergence phenomenon. Hence, we propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues, as well as achieve faster convergence in gradient-based optimization. In addition, a secure gradient sharing protocol using Homomorphic Encryption (HE) and Differential Privacy (DP) is introduced to defend against gradient leakage attack and avoid pairwise encryption that is not scalable. We demonstrate the proposed Federated Boosting (FedBoosting) method achieves noticeable improvements in both prediction accuracy and run-time efficiency in a visual text recognition task on public benchmark.
Code (2)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging
Federated learning, conducive to solving data privacy and security problems, has attracted increasing attention recently. However, the existing federated boosting model sequentially builds a decision tree model with the …
Federated LearningVertical Federated LearningGradient-less Federated Gradient Boosting Trees with Learnable Learning Rates
The privacy-sensitive nature of decentralized datasets and the robustness of eXtreme Gradient Boosting (XGBoost) on tabular data raise the needs to train XGBoost in the context of federated learning (FL). Existing works …
Federated LearningSecureCut: Federated Gradient Boosting Decision Trees with Efficient Machine Unlearning
In response to legislation mandating companies to honor the \textit{right to be forgotten} by erasing user data, it has become imperative to enable data removal in Vertical Federated Learning (VFL) where multiple parties…
Federated LearningMachine UnlearningVertical Federated LearningSecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree
Gradient boosting decision tree (GBDT) is an ensemble machine learning algorithm, which is widely used in industry, due to its good performance and easy interpretation. Due to the problem of data isolation and the requir…
Federated LearningPrivacy PreservingVertical Federated LearningM$^2$FGB: A Min-Max Gradient Boosting Framework for Subgroup Fairness
In recent years, fairness in machine learning has emerged as a critical concern to ensure that developed and deployed predictive models do not have disadvantageous predictions for marginalized groups. It is essential to …
Fairness