MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attacks
Implementations of SGD on distributed systems create new vulnerabilities, which can be identified and misused by one or more adversarial agents. Recently, it has been shown that well-known Byzantine-resilient gradient aggregation schemes are indeed vulnerable to informed attackers that can tailor the attacks (Fang et al., 2020; Xie et al., 2020b). We introduce MixTailor, a scheme based on randomization of the aggregation strategies that makes it impossible for the attacker to be fully informed. Deterministic schemes can be integrated into MixTailor on the fly without introducing any additional hyperparameters. Randomization decreases the capability of a powerful adversary to tailor its attacks, while the resulting randomized aggregation scheme is still competitive in terms of performance. For both iid and non-iid settings, we establish almost sure convergence guarantees that are both stronger and more general than those available in the literature. Our empirical studies across various datasets, attacks, and settings, validate our hypothesis and show that MixTailor successfully defends when well-known Byzantine-tolerant schemes fail.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MixNN: Protection of Federated Learning Against Inference Attacks by Mixing Neural Network Layers
Machine Learning (ML) has emerged as a core technology to provide learning models to perform complex tasks. Boosted by Machine Learning as a Service (MLaaS), the number of applications relying on ML capabilities is ever …
AttributeFederated LearningInference AttackPrivacy PreservingEncoded Gradients Aggregation against Gradient Leakage in Federated Learning
Federated learning enables isolated clients to train a shared model collaboratively by aggregating the locally-computed gradient updates. However, privacy information could be leaked from uploaded gradients and be expose…
Federated LearningG-FARS: Gradient-Field-based Auto-Regressive Sampling for 3D Part Grouping
This paper proposes a novel task named "3D part grouping". Suppose there is a mixed set containing scattered parts from various shapes. This task requires algorithms to find out every possible combination among all the p…
Graph Neural NetworkSFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
Recent mixed-policy optimization methods for LLM reasoning that interleave or blend supervised and reinforcement learning signals report improvements over the standard SFT-then-RL pipeline. We show that numerous recently…
Reinforcement LearningReveal Your Images: Gradient Leakage Attack against Unbiased Sampling-Based Secure Aggregation
Recently, some Unbiased Gradient Sampling-based (UGS) methods are proposed to mitigate gradient leakage by introducing sampling and unbiased transformation, such as MinMax Sampling in SIGMOD '22. In this paper, we first …
Federated Learning