Modelling Sentence Pairs with Tree-structured Attentive Encoder
We describe an attentive encoder that combines tree-structured recursive neural networks and sequential recurrent neural networks for modelling sentence pairs. Since existing attentive models exert attention on the sequential structure, we propose a way to incorporate attention into the tree topology. Specially, given a pair of sentences, our attentive encoder uses the representation of one sentence, which generated via an RNN, to guide the structural encoding of the other sentence on the dependency parse tree. We evaluate the proposed attentive encoder on three tasks: semantic similarity, paraphrase identification and true-false question selection. Experimental results show that our encoder outperforms all baselines and achieves state-of-the-art results on two tasks.
Code (1)
Tasks
Paraphrase IdentificationQuestion SelectionSemantic SimilaritySemantic Textual SimilaritySentenceSimilar Papers 제목 키워드 기반
Learning to Embed Sentences Using Attentive Recursive Trees
Sentence embedding is an effective feature representation for most deep learning-based NLP tasks. One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific st…
SentenceSentence EmbeddingSentence-EmbeddingAttentive Tree-structured Network for Monotonicity Reasoning
Many state-of-art neural models designed for monotonicity reasoning perform poorly on downward inference. To address this shortcoming, we developed an attentive tree-structured neural network. It consists of a tree-based…
Natural Language InferenceSentenceTree-Structured Neural Machine for Linguistics-Aware Sentence Generation
Different from other sequential data, sentences in natural language are structured by linguistic grammars. Previous generative conversational models with chain-structured decoder ignore this structure in human language a…
DecoderSentenceScalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding
Recent state-of-the-art natural language understanding models, such as BERT and XLNet, score a pair of sentences (A and B) using multiple cross-attention operations - a process in which each word in sentence A attends to…
Knowledge DistillationNatural Language UnderstandingSemantic SimilaritySentence+4Top-Down Tree Structured Text Generation
Text generation is a fundamental building block in natural language processing tasks. Existing sequential models performs autoregression directly over the text sequence and have difficulty generating long sentences of co…
SentenceText Generation