A Simple and Plug-and-play Method for Unsupervised Sentence Representation Enhancement
Generating proper embedding of sentences through an unsupervised way is beneficial to semantic matching and retrieval problems in real-world scenarios. This paper presents Representation ALchemy (RepAL), an extremely simple post-processing method that enhances sentence representations. The basic idea in RepAL is to de-emphasize redundant information of sentence embedding generated by pre-trained models. Through comprehensive experiments, we show that RepAL is free of training and is a plug-and-play method that can be combined with most existing unsupervised sentence learning models. We also conducted in-depth analysis to understand RepAL.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSentenceSentence EmbeddingSentence-EmbeddingSimilar Papers 제목 키워드 기반
RepAL: A Simple and Plug-and-play Method for Improving Unsupervised Sentence Representations
Unsupervised sentence representation learning is a fundamental problem in natural language processing and has been studied extensively in recent years. This paper presents Representation ALchemy (RepAL), an extremely sim…
Representation LearningSentenceSentence EmbeddingSentence-EmbeddingAlleviating Over-smoothing for Unsupervised Sentence Representation
Currently, learning better unsupervised sentence representations is the pursuit of many natural language processing communities. Lots of approaches based on pre-trained language models (PLMs) and contrastive learning hav…
Contrastive LearningSemantic Textual SimilaritySentenceUnsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence Filtering
Pre-trained Language Model (PLM) is nowadays the mainstay of Unsupervised Sentence Representation Learning (USRL). However, PLMs are sensitive to the frequency information of words from their pre-training corpora, result…
Language ModellingRepresentation LearningSentenceSentence EmbeddingsUnsupervised Document Embedding via Contrastive Augmentation
We present a contrasting learning approach with data augmentation techniques to learn document representations in an unsupervised manner. Inspired by recent contrastive self-supervised learning algorithms used for image …
Contrastive LearningData AugmentationDocument ClassificationDocument Embedding+3Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
The recent tremendous success of unsupervised word embeddings in a multitude of applications raises the obvious question if similar methods could be derived to improve embeddings (i.e. semantic representations) of word s…
SentenceSentence EmbeddingsWord Embeddings