Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention
We describe a span-level supervised attention loss that improves compositional generalization in semantic parsers. Our approach builds on existing losses that encourage attention maps in neural sequence-to-sequence models to imitate the output of classical word alignment algorithms. Where past work has used word-level alignments, we focus on spans; borrowing ideas from phrase-based machine translation, we align subtrees in semantic parses to spans of input sentences, and encourage neural attention mechanisms to mimic these alignments. This method improves the performance of transformers, RNNs, and structured decoders on three benchmarks of compositional generalization.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSemantic ParsingTranslationWord AlignmentSimilar Papers 제목 키워드 기반
Span-based Semantic Parsing for Compositional Generalization
Despite the success of sequence-to-sequence (seq2seq) models in semantic parsing, recent work has shown that they fail in compositional generalization, i.e., the ability to generalize to new structures built of component…
Semantic ParsingImproving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs). In this study, we empirically investigate…
Domain GeneralizationLanguage ModelingLanguage ModellingSemantic Parsing+2Towards Compositionally Generalizable Semantic Parsing in Large Language Models: A Survey
Compositional generalization is the ability of a model to generalize to complex, previously unseen types of combinations of entities from just having seen the primitives. This type of generalization is particularly relev…
Information RetrievalRetrievalSemantic ParsingSQL Parsing+2Making Transformers Solve Compositional Tasks
Several studies have reported the inability of Transformer models to generalize compositionally, a key type of generalization in many NLP tasks such as semantic parsing. In this paper we explore the design space of Trans…
Semantic ParsingMaking Transformers Solve Compositional Tasks
Several studies have reported the inability of Transformer models to generalize compositionally, a key type of generalization in many NLP tasks such as semantic parsing. In this paper we explore the design space of Trans…
Semantic Parsing