Hierarchical Poset Decoding for Compositional Generalization in Language
We formalize human language understanding as a structured prediction task where the output is a partially ordered set (poset). Current encoder-decoder architectures do not take the poset structure of semantics into account properly, thus suffering from poor compositional generalization ability. In this paper, we propose a novel hierarchical poset decoding paradigm for compositional generalization in language. Intuitively: (1) the proposed paradigm enforces partial permutation invariance in semantics, thus avoiding overfitting to bias ordering information; (2) the hierarchical mechanism allows to capture high-level structures of posets. We evaluate our proposed decoder on Compositional Freebase Questions (CFQ), a large and realistic natural language question answering dataset that is specifically designed to measure compositional generalization. Results show that it outperforms current decoders.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderQuestion AnsweringSemantic ParsingStructured PredictionSimilar Papers 제목 키워드 기반
Exclusive Hierarchical Decoding for Deep Keyphrase Generation
Keyphrase generation (KG) aims to summarize the main ideas of a document into a set of keyphrases. A new setting is recently introduced into this problem, in which, given a document, the model needs to predict a set of k…
DiversityKeyphrase GenerationGrounded Graph Decoding Improves Compositional Generalization in Question Answering
Question answering models struggle to generalize to novel compositions of training patterns, such to longer sequences or more complex test structures. Current end-to-end models learn a flat input embedding which can lose…
Question AnsweringRecursive Decoding: A Situated Cognition Approach to Compositional Generation in Grounded Language Understanding
Compositional generalization is a troubling blind spot for neural language models. Recent efforts have presented techniques for improving a model's ability to encode novel combinations of known inputs, but less work has …
DecoderHierarchical Partial-Order Models for Ranking
Rank aggregation combines information from ordered lists ranking items by preference. Classical parametric models for such data, including the Mallows and Plackett-Luce models, assume the orders concentrate around one or…
Bayesian InferenceForming Trees with Treeformers
Human language is known to exhibit a nested, hierarchical structure, allowing us to form complex sentences out of smaller pieces. However, many state-of-the-art neural networks models such as Transformers have no explici…
Abstractive Text SummarizationInductive BiasMachine TranslationNatural Language Understanding+1