Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings
We consider the problem of learning general-purpose, paraphrastic sentence embeddings, revisiting the setting of Wieting et al. (2016b). While they found LSTM recurrent networks to underperform word averaging, we present several developments that together produce the opposite conclusion. These include training on sentence pairs rather than phrase pairs, averaging states to represent sequences, and regularizing aggressively. These improve LSTMs in both transfer learning and supervised settings. We also introduce a new recurrent architecture, the Gated Recurrent Averaging Network, that is inspired by averaging and LSTMs while outperforming them both. We analyze our learned models, finding evidence of preferences for particular parts of speech and dependency relations.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence EmbeddingsTransfer LearningSimilar Papers 제목 키워드 기반
Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext
We consider the problem of learning general-purpose, paraphrastic sentence embeddings in the setting of Wieting et al. (2016b). We use neural machine translation to generate sentential paraphrases via back-translation of…
Machine TranslationSentenceSentence EmbeddingsTranslationTowards Universal Paraphrastic Sentence Embeddings
We consider the problem of learning general-purpose, paraphrastic sentence embeddings based on supervision from the Paraphrase Database (Ganitkevitch et al., 2013). We compare six compositional architectures, evaluating …
General ClassificationSentenceSentence EmbeddingsSentence Similarity+2Simple and Effective Paraphrastic Similarity from Parallel Translations
We present a model and methodology for learning paraphrastic sentence embeddings directly from bitext, removing the time-consuming intermediate step of creating paraphrase corpora. Further, we show that the resulting mod…
SentenceSentence EmbeddingsA Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations
We propose a generative model for a sentence that uses two latent variables, with one intended to represent the syntax of the sentence and the other to represent its semantics. We show we can achieve better disentangleme…
DisentanglementSemantic SimilaritySemantic Textual SimilaritySentenceParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations
We describe PARANMT-50M, a dataset of more than 50 million English-English sentential paraphrase pairs. We generated the pairs automatically by using neural machine translation to translate the non-English side of a larg…
Machine TranslationNatural Language UnderstandingParaphrase GenerationSemantic Textual Similarity+3