Compositional Generalization by Factorizing Alignment and Translation
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in cognitive science suggesting a functional distinction between systems for syntactic and semantic processing, we implement a modification to an existing approach in neural machine translation, imposing an analogous separation between alignment and translation. The resulting architecture substantially outperforms standard recurrent networks on the SCAN dataset, a compositional generalization task, without any additional supervision. Our work suggests that learning to align and to translate in separate modules may be a useful heuristic for capturing compositional structure.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSystematic GeneralizationTranslationSimilar Papers 제목 키워드 기반
Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention
We describe a span-level supervised attention loss that improves compositional generalization in semantic parsers. Our approach builds on existing losses that encourage attention maps in neural sequence-to-sequence model…
Machine TranslationSemantic ParsingTranslationWord AlignmentRevisiting Iterative Back-Translation from the Perspective of Compositional Generalization
Human intelligence exhibits compositional generalization (i.e., the capacity to understand and produce unseen combinations of seen components), but current neural seq2seq models lack such ability. In this paper, we revis…
TranslationTranslate First Reorder Later: Leveraging Monotonicity in Semantic Parsing
Prior work in semantic parsing has shown that conventional seq2seq models fail at compositional generalization tasks. This limitation led to a resurgence of methods that model alignments between sentences and their corre…
Semantic ParsingEvaluating Structural Generalization in Neural Machine Translation
Compositional generalization refers to the ability to generalize to novel combinations of previously observed words and syntactic structures. Since it is regarded as a desired property of neural models, recent work has a…
Machine TranslationSemantic ParsingSentenceTranslationOn Compositional Generalization of Neural Machine Translation
Modern neural machine translation (NMT) models have achieved competitive performance in standard benchmarks such as WMT. However, there still exist significant issues such as robustness, domain generalization, etc. In th…
Domain GeneralizationMachine TranslationNMTSentence+1