Word Embedding Dimension Reduction via Weakly-Supervised Feature Selection
As a fundamental task in natural language processing, word embedding converts each word into a representation in a vector space. A challenge with word embedding is that as the vocabulary grows, the vector space's dimension increases, which can lead to a vast model size. Storing and processing word vectors are resource-demanding, especially for mobile edge-devices applications. This paper explores word embedding dimension reduction. To balance computational costs and performance, we propose an efficient and effective weakly-supervised feature selection method named WordFS. It has two variants, each utilizing novel criteria for feature selection. Experiments on various tasks (e.g., word and sentence similarity and binary and multi-class classification) indicate that the proposed WordFS model outperforms other dimension reduction methods at lower computational costs. We have released the code for reproducibility along with the paper.
Code (1)
Tasks
Dimensionality Reductionfeature selectionMulti-class ClassificationSentenceSentence SimilarityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Simple and Effective Dimensionality Reduction for Word Embeddings
Word embeddings have become the basic building blocks for several natural language processing and information retrieval tasks. Pre-trained word embeddings are used in several downstream applications as well as for constr…
Dimensionality ReductionInformation RetrievalRetrievalWord Embeddings+1Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis
Unsupervised learned representations of polysemous words generate a large of pseudo multi senses since unsupervised methods are overly sensitive to contextual variations. In this paper, we address the pseudo multi-sense …
Dimensionality ReductionWord EmbeddingsWord SimilarityFactor Augmented Supervised Learning with Text Embeddings
Large language models (LLMs) generate text embeddings from text data, producing vector representations that capture the semantic meaning and contextual relationships of words. However, the high dimensionality of these em…
Anomaly DetectionAn Uncertainty-Aware Encoder for Aspect Detection
Aspect detection is a fundamental task in opinion mining. Previous works use seed words either as priors of topic models, as anchors to guide the learning of aspects, or as features of aspect classifiers. This paper pres…
Opinion MiningTopic ModelsSLICE: Supersense-based Lightweight Interpretable Contextual Embeddings
Contextualised embeddings such as BERT have become de facto state-of-the-art references in many NLP applications, thanks to their impressive performances. However, their opaqueness makes it hard to interpret their behavi…