Model Reconstruction from Model Explanations
We show through theory and experiment that gradient-based explanations of a model quickly reveal the model itself. Our results speak to a tension between the desire to keep a proprietary model secret and the ability to offer model explanations. On the theoretical side, we give an algorithm that provably learns a two-layer ReLU network in a setting where the algorithm may query the gradient of the model with respect to chosen inputs. The number of queries is independent of the dimension and nearly optimal in its dependence on the model size. Of interest not only from a learning-theoretic perspective, this result highlights the power of gradients rather than labels as a learning primitive. Complementing our theory, we give effective heuristics for reconstructing models from gradient explanations that are orders of magnitude more query-efficient than reconstruction attacks relying on prediction interfaces.
Code (0)
등록된 구현이 없습니다.
Tasks
modelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ReconXF: Graph Reconstruction Attack via Public Feature Explanations on Privatized Node Features and Labels
Graph Neural Networks (GNNs) achieve high performance across many applications but function as black-box models, limiting their use in critical domains like healthcare and criminal justice. Explainability methods address…
DenoisingGraph ReconstructionReconstruction AttackUnification-based Reconstruction of Multi-hop Explanations for Science Questions
This paper presents a novel framework for reconstructing multi-hop explanations in science Question Answering (QA). While existing approaches for multi-hop reasoning build explanations considering each question in isolat…
ClusteringInformation RetrievalQuestion AnsweringRetrieval+1Private Graph Extraction via Feature Explanations
Privacy and interpretability are two important ingredients for achieving trustworthy machine learning. We study the interplay of these two aspects in graph machine learning through graph reconstruction attacks. The goal …
BIG-bench Machine LearningGraph ReconstructionModel Reconstruction Using Counterfactual Explanations: A Perspective From Polytope Theory
Counterfactual explanations provide ways of achieving a favorable model outcome with minimum input perturbation. However, counterfactual explanations can also be leveraged to reconstruct the model by strategically traini…
counterfactualModel extractionGraph Reconstruction from Differentially Private GNN Explanations
Regulatory frameworks such as GDPR increasingly require that ML predictions be accompanied by post-hoc explanations, even when raw data and trained models cannot be released. Differential privacy (DP) is the standard mit…