Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning
Deep compositional models of meaning acting on distributional representations of words in order to produce vectors of larger text constituents are evolving to a popular area of NLP research. We detail a compositional distributional framework based on a rich form of word embeddings that aims at facilitating the interactions between words in the context of a sentence. Embeddings and composition layers are jointly learned against a generic objective that enhances the vectors with syntactic information from the surrounding context. Furthermore, each word is associated with a number of senses, the most plausible of which is selected dynamically during the composition process. We evaluate the produced vectors qualitatively and quantitatively with positive results. At the sentence level, the effectiveness of the framework is demonstrated on the MSRPar task, for which we report results within the state-of-the-art range.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
Cross-lingual Word Sense Disambiguation using mBERT Embeddings with Syntactic Dependencies
Cross-lingual word sense disambiguation (WSD) tackles the challenge of disambiguating ambiguous words across languages given context. The pre-trained BERT embedding model has been proven to be effective in extracting con…
Word EmbeddingsWord Sense DisambiguationSensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings
Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored thepotential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct…
Word EmbeddingsTowards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings
Cross-lingual word embeddings (CLWE) have been proven useful in many cross-lingual tasks. However, most existing approaches to learn CLWE including the ones with contextual embeddings are sense agnostic. In this work, we…
Cross-Lingual NERCross-Lingual Word EmbeddingsNERSentiment Analysis+3Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of Arabic Dialects
Arabic sentiment analysis models have employed compositional embedding features to represent the Arabic dialectal content. These embeddings are usually composed via ordered, syntax-aware composition functions and learned…
Arabic Sentiment AnalysisSentiment AnalysisBreaking Down Word Semantics from Pre-trained Language Models through Layer-wise Dimension Selection
Contextual word embeddings obtained from pre-trained language model (PLM) have proven effective for various natural language processing tasks at the word level. However, interpreting the hidden aspects within embeddings,…
Binary ClassificationLanguage ModelingLanguage ModellingRepresentation Learning+1