Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in Image and Signal processing suggest that wavelets can be effectively applied to Natural Language Processing (NLP) tasks that capture a variety of linguistic properties. In this paper, we leverage the power of applying Discrete Wavelet Transforms (DWT) to word and sentence embeddings. We first evaluate, intrinsically and extrinsically, how wavelets can effectively be used to consolidate important information in a word vector while reducing its dimensionality. We further combine DWT with Discrete Cosine Transform (DCT) to propose a non-parameterized model that compresses a sentence with a dense amount of information in a fixed size vector based on locally varying word features. We show the efficacy of the proposed paradigm on downstream applications models yielding comparable and even superior (in some tasks) results to original embeddings.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Discrete Wavelet Transform for Efficient Word Embeddings and Sentence Encoding
Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in image and signal processing suggest that wavelets can be effectively applied to Natural Language Processi…
SentenceSentence EmbeddingsWord EmbeddingsSemantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
Wavelet transforms, a powerful mathematical tool, have been widely used in different domains, including Signal and Image processing, to unravel intricate patterns, enhance data representation, and extract meaningful feat…
Semantic SimilarityCompression, Restoration, Re-sampling, Compressive Sensing: Fast Transforms in Digital Imaging
Transform image processing methods are methods that work in domains of image transforms, such as Discrete Fourier, Discrete Cosine, Wavelet and alike. They are the basic tool in image compression, in image restoration, i…
Compressive SensingImage CompressionImage ReconstructionImage RestorationSpectral Data Augmentation Techniques to quantify Lung Pathology from CT-images
Data augmentation is of paramount importance in biomedical image processing tasks, characterized by inadequate amounts of labelled data, to best use all of the data that is present. In-use techniques range from intensity…
Data AugmentationTexture ClassificationUse of Deterministic Transforms to Design Weight Matrices of a Neural Network
Self size-estimating feedforward network (SSFN) is a feedforward multilayer network. For the existing SSFN, a part of each weight matrix is trained using a layer-wise convex optimization approach (a supervised training),…