Sentence embedding with contrastive multi-views learning
In this work, we propose a self-supervised method to learn sentence representations with an injection of linguistic knowledge. Multiple linguistic frameworks propose diverse sentence structures from which semantic meaning might be expressed out of compositional words operations. We aim to take advantage of this linguist diversity and learn to represent sentences by contrasting these diverse views. Formally, multiple views of the same sentence are mapped to close representations. On the contrary, views from other sentences are mapped further. By contrasting different linguistic views, we aim at building embeddings which better capture semantic and which are less sensitive to the sentence outward form.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversitySentenceSentence EmbeddingSentence-EmbeddingSimilar Papers 제목 키워드 기반
miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings
This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the state-of-the-art in few-shot sentence embedding. The proposed approach imposes alignment between the at…
Contrastive LearningFew-Shot LearningSelf-Supervised LearningSentence+3InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings
Contrastive learning has been extensively studied in sentence embedding learning, which assumes that the embeddings of different views of the same sentence are closer. The constraint brought by this assumption is weak, a…
Contrastive LearningLanguage ModelingLanguage ModellingRepresentation Learning+6Contrasting distinct structured views to learn sentence embeddings
We propose a self-supervised method that builds sentence embeddings from the combination of diverse explicit syntactic structures of a sentence. We assume structure is crucial to build consistent representations as we e…
SentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsSentence Embeddings using Supervised Contrastive Learning
Sentence embeddings encode sentences in fixed dense vectors and have played an important role in various NLP tasks and systems. Methods for building sentence embeddings include unsupervised learning such as Quick-Thought…
Contrastive LearningSemantic Textual SimilaritySentenceSentence Embeddings+1Improving Multi-lingual Alignment Through Soft Contrastive Learning
Making decent multi-lingual sentence representations is critical to achieve high performances in cross-lingual downstream tasks. In this work, we propose a novel method to align multi-lingual embeddings based on the simi…
Contrastive LearningSentenceSTS