PrivateRec: Differentially Private Training and Serving for Federated News Recommendation
Collecting and training over sensitive personal data raise severe privacy concerns in personalized recommendation systems, and federated learning can potentially alleviate the problem by training models over decentralized user data.However, a theoretically private solution in both the training and serving stages of federated recommendation is essential but still lacking.Furthermore, naively applying differential privacy (DP) to the two stages in federated recommendation would fail to achieve a satisfactory trade-off between privacy and utility due to the high-dimensional characteristics of model gradients and hidden representations.In this work, we propose a federated news recommendation method for achieving a better utility in model training and online serving under a DP guarantee.We first clarify the DP definition over behavior data for each round in the life-circle of federated recommendation systems.Next, we propose a privacy-preserving online serving mechanism under this definition based on the idea of decomposing user embeddings with public basic vectors and perturbing the lower-dimensional combination coefficients. We apply a random behavior padding mechanism to reduce the required noise intensity for better utility. Besides, we design a federated recommendation model training method, which can generate effective and public basic vectors for serving while providing DP for training participants. We avoid the dimension-dependent noise for large models via label permutation and differentially private attention modules. Experiments on real-world news recommendation datasets validate that our method achieves superior utility under a DP guarantee in both training and serving of federated news recommendations.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningNews RecommendationPrivacy PreservingRecommendation SystemsSimilar Papers 제목 키워드 기반
Differentially Private Federated Learning: A Systematic Review
In recent years, privacy and security concerns in machine learning have promoted trusted federated learning to the forefront of research. Differential privacy has emerged as the de facto standard for privacy protection i…
Federated LearningPrivacy PreservingShuffled Differentially Private Federated Learning for Time Series Data Analytics
Trustworthy federated learning aims to achieve optimal performance while ensuring clients' privacy. Existing privacy-preserving federated learning approaches are mostly tailored for image data, lacking applications for t…
Activity RecognitionFederated LearningHuman Activity RecognitionPrivacy Preserving+1Privately Customizing Prefinetuning to Better Match User Data in Federated Learning
In Federated Learning (FL), accessing private client data incurs communication and privacy costs. As a result, FL deployments commonly prefinetune pretrained foundation models on a (large, possibly public) dataset that i…
Federated LearningLanguage ModelingLanguage ModellingLarge Language Model+1Learning Rate Adaptation for Federated and Differentially Private Learning
We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: …
Federated LearningHybrid Differentially Private Federated Learning on Vertically Partitioned Data
We present HDP-VFL, the first hybrid differentially private (DP) framework for vertical federated learning (VFL) to demonstrate that it is possible to jointly learn a generalized linear model (GLM) from vertically partit…
Federated LearningPrivacy PreservingVertical Federated Learning