Private Multi-Task Learning: Formulation and Applications to Federated Learning
Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications in areas such as healthcare, finance, and IoT computing, where sensitive data from multiple, varied sources are shared for the purpose of learning. In this work, we formalize notions of client-level privacy for MTL via joint differential privacy (JDP), a relaxation of differential privacy for mechanism design and distributed optimization. We then propose an algorithm for mean-regularized MTL, an objective commonly used for applications in personalized federated learning, subject to JDP. We analyze our objective and solver, providing certifiable guarantees on both privacy and utility. Empirically, we find that our method provides improved privacy/utility trade-offs relative to global baselines across common federated learning benchmarks.
Code (1)
Tasks
BIG-bench Machine LearningDistributed OptimizationFederated LearningMulti-Task LearningPersonalized Federated LearningSimilar Papers 제목 키워드 기반
Federated Multi-Objective Learning
In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centr…
Federated LearningMulti-Task LearningFederated Learning of a Mixture of Global and Local Models
We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the…
Federated LearningFederated Estimation of Causal Effects from Observational Data
Many modern applications collect data that comes in federated spirit, with data kept locally and undisclosed. Till date, most insight into the causal inference requires data to be stored in a central repository. We prese…
Causal InferenceGaussian ProcessesSurvey of Personalization Techniques for Federated Learning
Federated learning enables machine learning models to learn from private decentralized data without compromising privacy. The standard formulation of federated learning produces one shared model for all clients. Statisti…
BIG-bench Machine LearningFederated LearningSurveyFadMan: Federated Anomaly Detection across Multiple Attributed Networks
Anomaly subgraph detection has been widely used in various applications, ranging from cyber attack in computer networks to malicious activities in social networks. Despite an increasing need for federated anomaly detecti…
Anomaly DetectionData IntegrationFederated LearningVertical Federated Learning