Better Early than Late: Fusing Topics with Word Embeddings for Neural Question Paraphrase Identification
Question paraphrase identification is a key task in Community Question Answering (CQA) to determine if an incoming question has been previously asked. Many current models use word embeddings to identify duplicate questions, but the use of topic models in feature-engineered systems suggests that they can be helpful for this task, too. We therefore propose two ways of merging topics with word embeddings (early vs. late fusion) in a new neural architecture for question paraphrase identification. Our results show that our system outperforms neural baselines on multiple CQA datasets, while an ablation study highlights the importance of topics and especially early topic-embedding fusion in our architecture.
Code (0)
등록된 구현이 없습니다.
Tasks
Community Question AnsweringParaphrase IdentificationQuestion AnsweringTopic ModelsWord EmbeddingsSimilar Papers 제목 키워드 기반
Fusing finetuned models for better pretraining
Pretrained models are the standard starting point for training. This approach consistently outperforms the use of a random initialization. However, pretraining is a costly endeavour that few can undertake. In this paper,…
A Latent Source Model for Nonparametric Time Series Classification
For classifying time series, a nearest-neighbor approach is widely used in practice with performance often competitive with or better than more elaborate methods such as neural networks, decision trees, and support vecto…
ClassificationGeneral ClassificationTime SeriesTime Series Analysis+1Topic Modeling with Wasserstein Autoencoders
We propose a novel neural topic model in the Wasserstein autoencoders (WAE) framework. Unlike existing variational autoencoder based models, we directly enforce Dirichlet prior on the latent document-topic vectors. We ex…
DiversityGenerative Adversarial NetworkTopic ModelsCovid-Transformer: Detecting COVID-19 Trending Topics on Twitter Using Universal Sentence Encoder
The novel corona-virus disease (also known as COVID-19) has led to a pandemic, impacting more than 200 countries across the globe. With its global impact, COVID-19 has become a major concern of people almost everywhere, …
ClusteringSentenceSentence SimilarityText SummarizationTopic Modeling Using Distributed Word Embeddings
We propose a new algorithm for topic modeling, Vec2Topic, that identifies the main topics in a corpus using semantic information captured via high-dimensional distributed word embeddings. Our technique is unsupervised an…
Word Embeddings