Streaming word similarity mining on the cheap
Accurately and efficiently estimating word similarities from text is fundamental in natural language processing. In this paper, we propose a fast and lightweight method for estimating similarities from streams by explicitly counting second-order co-occurrences. The method rests on the observation that words that are highly correlated with respect to such counts are also highly similar with respect to first-order co-occurrences. Using buffers of co-occurred words per word to count second-order co-occurrences, we can then estimate similarities in a single pass over data without having to do prohibitively expensive similarity calculations. We demonstrate that this approach is scalable, converges rapidly, behaves robustly under parameter changes, and that it captures word similarities on par with those given by state-of-the-art word embeddings.
Code (0)
등록된 구현이 없습니다.
Tasks
Document ClassificationWord AlignmentWord EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
Streaming Word Embeddings with the Space-Saving Algorithm
We develop a streaming (one-pass, bounded-memory) word embedding algorithm based on the canonical skip-gram with negative sampling algorithm implemented in word2vec. We compare our streaming algorithm to word2vec empiric…
Word EmbeddingsCTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting
This paper introduces a novel approach for streaming openvocabulary keyword spotting (KWS) with text-based keyword enrollment. For every input frame, the proposed method finds the optimal alignment ending at the frame us…
Keyword SpottingStreaming T5-based Text-to-Speech Synthesis with Limited Lookahead
Streaming text-to-speech synthesis in cascaded LLM-TTS systems still faces latency challenges as most TTS models require full context before initiating generation. We present S5-TTS, a streaming variant of T5-TTS that en…
Text-To-Speech SynthesisWord net based Method for Determining Semantic Sentence Similarity through various Word Senses
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition
Recently, there has been an increasing interest in two-pass streaming end-to-end speech recognition (ASR) that incorporates a 2nd-pass rescoring model on top of the conventional 1st-pass streaming ASR model to improve re…
speech-recognitionSpeech Recognition