One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations
Sentence embedding methods have made remarkable progress, yet they still struggle to capture the implicit semantics within sentences. This can be attributed to the inherent limitations of conventional sentence embedding methods that assign only a single vector per sentence. To overcome this limitation, we propose DualCSE, a sentence embedding method that assigns two embeddings to each sentence: one representing the explicit semantics and the other representing the implicit semantics. These embeddings coexist in the shared space, enabling the selection of the desired semantics for specific purposes such as information retrieval and text classification. Experimental results demonstrate that DualCSE can effectively encode both explicit and implicit meanings and improve the performance of the downstream task.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalContrastive LearningText ClassificationSimilar Papers 제목 키워드 기반
TransAug: Translate as Augmentation for Sentence Embeddings
While contrastive learning greatly advances the representation of sentence embeddings, it is still limited by the size of the existing sentence datasets. In this paper, we present TransAug (Translate as Augmentation), wh…
Contrastive LearningData AugmentationSemantic Textual SimilaritySentence+2AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge
Generic sentence embeddings provide a coarse-grained approximation of semantic textual similarity but ignore specific aspects that make texts similar. Conversely, aspect-based sentence embeddings provide similarities bet…
Contrastive LearningInformation RetrievalRetrievalSemantic Textual Similarity+4SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens
The verbosity of Chain-of-Thought (CoT) reasoning hinders its mass deployment in efficiency-critical applications. Recently, implicit CoT approaches have emerged, which encode reasoning steps within LLM's hidden embeddin…
Knowledge DistillationContrasting distinct structured views to learn sentence embeddings
We propose a self-supervised method that builds sentence embeddings from the combination of diverse explicit syntactic structures of a sentence. We assume structure is crucial to build consistent representations as we e…
SentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsContrastive Learning of Sentence Representations
Learning sentence representations which capture rich semantic meanings has been crucial for many NLP tasks. Pre-trained language models such as BERT have achieved great success in NLP, but sentence embeddings extracted d…
Contrastive LearningSemantic SimilaritySemantic Textual SimilaritySentence+1