Static Fuzzy Bag-of-Words: a lightweight sentence embedding algorithm
The introduction of embedding techniques has pushed forward significantly the Natural Language Processing field. Many of the proposed solutions have been presented for word-level encoding; anyhow, in the last years, new mechanism to treat information at an higher level of aggregation, like at sentence- and document-level, have emerged. With this work we address specifically the sentence embeddings problem, presenting the Static Fuzzy Bag-of-Word model. Our model is a refinement of the Fuzzy Bag-of-Words approach, providing sentence embeddings with a predefined dimension. SFBoW provides competitive performances in Semantic Textual Similarity benchmarks, while requiring low computational resources.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic Textual SimilaritySentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsSimilar Papers 제목 키워드 기반
Static Fuzzy Bag-of-Words: a Lightweight and Fast Sentence Embedding Algorithm
Static Word Embeddings for Sentence Semantic Representation
We propose new static word embeddings optimised for sentence semantic representation. We first extract word embeddings from a pre-trained Sentence Transformer, and improve them with sentence-level principal component ana…
Contrastive LearningKnowledge DistillationSentenceWord EmbeddingsAnalysis of Word Embeddings Using Fuzzy Clustering
In data dominated systems and applications, a concept of representing words in a numerical format has gained a lot of attention. There are a few approaches used to generate such a representation. An interesting issue tha…
ClusteringSemantic SimilaritySemantic Textual SimilarityWord EmbeddingsOne Word Is Not Enough: Simple Prompts Improve Word Embeddings
Text embedding models are designed for sentence-level applications like retrieval and semantic similarity, and are primarily evaluated on sentence-level benchmarks. Their behavior on isolated words is less understood. We…
Semantic SimilarityWord SimilaritySentence Curve Language Models
Language models (LMs) are a central component of modern AI systems, and diffusion language models (DLMs) have recently emerged as a competitive alternative. Both paradigms rely on word embeddings not only to represent th…
Knowledge Distillation