On Robustness of Neural Semantic Parsers
Semantic parsing maps natural language (NL) utterances into logical forms (LFs), which underpins many advanced NLP problems. Semantic parsers gain performance boosts with deep neural networks, but inherit vulnerabilities against adversarial examples. In this paper, we provide the empirical study on the robustness of semantic parsers in the presence of adversarial attacks. Formally, adversaries of semantic parsing are considered to be the perturbed utterance-LF pairs, whose utterances have exactly the same meanings as the original ones. A scalable methodology is proposed to construct robustness test sets based on existing benchmark corpora. Our results answered five research questions in measuring the sate-of-the-art parsers' performance on robustness test sets, and evaluating the effect of data augmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationSemantic ParsingSimilar Papers 제목 키워드 기반
Exploring the Robustness of Human Parsers Towards Common Corruptions
Human parsing aims to segment each pixel of the human image with fine-grained semantic categories. However, current human parsers trained with clean data are easily confused by numerous image corruptions such as blur and…
Data AugmentationDiversityHuman ParsingSemantic SegmentationOn Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex
Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advancements in few-shot language models trained on code have demonstrated superior …
Adversarial RobustnessLanguage ModelingLanguage ModellingSemantic ParsingFast Domain Adaptation of Semantic Parsers via Paraphrase Attention
Semantic parsers are used to convert user{'}s natural language commands to executable logical form in intelligent personal agents. Labeled datasets required to train such parsers are expensive to collect, and are never c…
Domain AdaptationIs neural semantic parsing good at ellipsis resolution, or isn't it?
Neural semantic parsers have shown good overall performance for a variety of linguistic phenomena, reaching semantic matching scores of more than 90%. But how do such parsers perform on strongly context-sensitive phenome…
Data AugmentationSemantic ParsingEvaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples
Despite achieving prominent performance on many important tasks, it has been reported that neural networks are vulnerable to adversarial examples. Previously studies along this line mainly focused on semantic tasks such …
Dependency ParsingQuestion AnsweringReading ComprehensionSentence+1