Embeddings in Natural Language Processing
Embeddings have been one of the most important topics of interest in NLP for the past decade. Representing knowledge through a low-dimensional vector which is easily integrable in modern machine learning models has played a central role in the development of the field. Embedding techniques initially focused on words but the attention soon started to shift to other forms. This tutorial will provide a high-level synthesis of the main embedding techniques in NLP, in the broad sense. We will start by conventional word embeddings (e.g., Word2Vec and GloVe) and then move to other types of embeddings, such as sense-specific and graph alternatives. We will finalize with an overview of the trending contextualized representations (e.g., ELMo and BERT) and explain their potential and impact in NLP.
Code (0)
등록된 구현이 없습니다.
Tasks
High-Level SynthesisWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Effectiveness of Pre-Trained Code Embeddings
Word embeddings are widely used in machine learning based natural language processing systems. It is common to use pre-trained word embeddings which provide benefits such as reduced training time and improved overall per…
Extreme SummarizationWord EmbeddingsSupervised Contextual Embeddings for Transfer Learning in Natural Language Processing Tasks
Pre-trained word embeddings are the primary method for transfer learning in several Natural Language Processing (NLP) tasks. Recent works have focused on using unsupervised techniques such as language modeling to obtain …
Language ModelingLanguage ModellingTransfer LearningWord EmbeddingsEmbeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP
Transfer learning, particularly approaches that combine multi-task learning with pre-trained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years.…
Dependency ParsingLanguage ModelingLanguage ModellingMasked Language Modeling+5emoji2vec: Learning Emoji Representations from their Description
Many current natural language processing applications for social media rely on representation learning and utilize pre-trained word embeddings. There currently exist several publicly-available, pre-trained sets of word e…
Representation LearningSentiment AnalysisWord Embeddings