InferLite: Simple Universal Sentence Representations from Natural Language Inference Data
Natural language inference has been shown to be an effective supervised task for learning generic sentence embeddings. In order to better understand the components that lead to effective representations, we propose a lightweight version of InferSent, called InferLite, that does not use any recurrent layers and operates on a collection of pre-trained word embeddings. We show that a simple instance of our model that makes no use of context, word ordering or position can still obtain competitive performance on the majority of downstream prediction tasks, with most performance gaps being filled by adding local contextual information through temporal convolutions. Our models can be trained in under 1 hour on a single GPU and allows for fast inference of new representations. Finally we describe a semantic hashing layer that allows our model to learn generic binary codes for sentences.
Code (0)
등록된 구현이 없습니다.
Tasks
GPUNatural Language InferencePositionSentenceSentence EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
SufiSent - Universal Sentence Representations Using Suffix Encodings
Computing universal distributed representations of sentences is a fundamental task in natural language processing. We propose a method to learn such representations by encoding the suffixes of word sequences in a sentenc…
Natural Language InferenceSentenceSentEval: An Evaluation Toolkit for Universal Sentence Representations
We introduce SentEval, a toolkit for evaluating the quality of universal sentence representations. SentEval encompasses a variety of tasks, including binary and multi-class classification, natural language inference and …
General ClassificationMulti-class ClassificationNatural Language InferenceSentence+1Probing Contextualized Sentence Representations with Visual Awareness
We present a universal framework to model contextualized sentence representations with visual awareness that is motivated to overcome the shortcomings of the multimodal parallel data with manual annotations. For each sen…
DiversityMachine TranslationNatural Language InferenceSentence+1Unsupervised Learning of Sentence Representations Using Sequence Consistency
Computing universal distributed representations of sentences is a fundamental task in natural language processing. We propose ConsSent, a simple yet surprisingly powerful unsupervised method to learn such representations…
FormSentenceTransfer LearningSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however no…
Cross-Lingual Natural Language InferenceNatural Language InferenceSemantic Textual SimilaritySentence+2