What Learned Representations and Influence Functions Can Tell Us About Adversarial Examples
Adversarial examples, deliberately crafted using small perturbations to fool deep neural networks, were first studied in image processing and more recently in NLP. While approaches to detecting adversarial examples in NLP have largely relied on search over input perturbations, image processing has seen a range of techniques that aim to characterise adversarial subspaces over the learned representations. In this paper, we adapt two such approaches to NLP, one based on nearest neighbors and influence functions and one on Mahalanobis distances. The former in particular produces a state-of-the-art detector when compared against several strong baselines; moreover, the novel use of influence functions provides insight into how the nature of adversarial example subspaces in NLP relate to those in image processing, and also how they differ depending on the kind of NLP task.
Code (2)
Similar Papers 제목 키워드 기반
Where Did Your Model Learn That? Label-free Influence for Self-supervised Learning
Self-supervised learning (SSL) has revolutionized learning from large-scale unlabeled datasets, yet the intrinsic relationship between pretraining data and the learned representations remains poorly understood. Tradition…
FairnessSelf-Supervised LearningIf Influence Functions are the Answer, Then What is the Question?
Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters. While influence estimates align well with leave-one-out retraining for linear models, recent w…
What Do GNNs Actually Learn? Towards Understanding their Representations
In recent years, graph neural networks (GNNs) have achieved great success in the field of graph representation learning. Although prior work has shed light on the expressiveness of those models (\ie whether they can dist…
Graph Representation LearningRepresentation LearningDecoupling Learning Rules from Representations
In the artificial intelligence field, learning often corresponds to changing the parameters of a parameterized function. A learning rule is an algorithm or mathematical expression that specifies precisely how the paramet…
Reinforcement LearningDivergent representations of ethological visual inputs emerge from supervised, unsupervised, and reinforcement learning
Artificial neural systems trained using reinforcement, supervised, and unsupervised learning all acquire internal representations of high dimensional input. To what extent these representations depend on the different le…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Transfer Learning