Robust Textual Embedding against Word-level Adversarial Attacks
We attribute the vulnerability of natural language processing models to the fact that similar inputs are converted to dissimilar representations in the embedding space, leading to inconsistent outputs, and we propose a novel robust training method, termed Fast Triplet Metric Learning (FTML). Specifically, we argue that the original sample should have similar representation with its adversarial counterparts and distinguish its representation from other samples for better robustness. To this end, we adopt the triplet metric learning into the standard training to pull words closer to their positive samples (i.e., synonyms) and push away their negative samples (i.e., non-synonyms) in the embedding space. Extensive experiments demonstrate that FTML can significantly promote the model robustness against various advanced adversarial attacks while keeping competitive classification accuracy on original samples. Besides, our method is efficient as it only needs to adjust the embedding and introduces very little overhead on the standard training. Our work shows great potential of improving the textual robustness through robust word embedding.
Code (1)
Tasks
AttributeMetric LearningTripletSimilar Papers 제목 키워드 기반
Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attacks
The language models, especially the basic text classification models, have been shown to be susceptible to textual adversarial attacks such as synonym substitution and word insertion attacks. To defend against such attac…
text-classificationText ClassificationLanguage Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation
Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work…
Language ModellingLEMMAWord EmbeddingsWord Sense DisambiguationFast Adversarial Training against Textual Adversarial Attacks
Many adversarial defense methods have been proposed to enhance the adversarial robustness of natural language processing models. However, most of them introduce additional pre-set linguistic knowledge and assume that the…
Adversarial DefenseAdversarial RobustnessA Simple General Method for Detecting Textual Adversarial Examples
Although deep neural networks have achieved state-of-the-art performance in various machine learning and artificial intelligence tasks, adversarial examples, constructed by adding small non-random perturbations to correc…
Ensemble LearningRepresentation LearningSentenceText Adversarial Purification as Defense against Adversarial Attacks
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack. Generally, adversarial purification aims to remove the adversarial pe…
Adversarial AttackAdversarial DefenseAdversarial Purification