Robust Word Vectors: Context-Informed Embeddings for Noisy Texts
We suggest a new language-independent architecture of robust word vectors (RoVe). It is designed to alleviate the issue of typos, which are common in almost any user-generated content, and hinder automatic text processing. Our model is morphologically motivated, which allows it to deal with unseen word forms in morphologically rich languages. We present the results on a number of Natural Language Processing (NLP) tasks and languages for the variety of related architectures and show that proposed architecture is typo-proof.
Code (0)
등록된 구현이 없습니다.
Tasks
Morphological AnalysisWord EmbeddingsSimilar Papers 제목 키워드 기반
Lexical Relation Mining in Neural Word Embeddings
Work with neural word embeddings and lexical relations has largely focused on confirmatory experiments which use human-curated examples of semantic and syntactic relations to validate against. In this paper, we explore t…
RelationWord EmbeddingsContext Vectors are Reflections of Word Vectors in Half the Dimensions
This paper takes a step towards theoretical analysis of the relationship between word embeddings and context embeddings in models such as word2vec. We start from basic probabilistic assumptions on the nature of word vect…
Text GenerationWord EmbeddingsDeriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection
One of the long-standing challenges in lexical semantics consists in learning representations of words which reflect their semantic properties. The remarkable success of word embeddings for this purpose suggests that hig…
SentenceWord EmbeddingsModelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings
While the success of pre-trained language models has largely eliminated the need for high-quality static word vectors in many NLP applications, such vectors continue to play an important role in tasks where words need to…
Knowledge Base CompletionEstimator Vectors: OOV Word Embeddings based on Subword and Context Clue Estimates
Semantic representations of words have been successfully extracted from unlabeled corpuses using neural network models like word2vec. These representations are generally high quality and are computationally inexpensive t…
Word Embeddings