Transformer-based dimensionality reduction
Recently, Transformer is much popular and plays an important role in the fields of Machine Learning (ML), Natural Language Processing (NLP), and Computer Vision (CV), etc. In this paper, based on the Vision Transformer (ViT) model, a new dimensionality reduction (DR) model is proposed, named Transformer-DR. From data visualization, image reconstruction and face recognition, the representation ability of Transformer-DR after dimensionality reduction is studied, and it is compared with some representative DR methods to understand the difference between Transformer-DR and existing DR methods. The experimental results show that Transformer-DR is an effective dimensionality reduction method.
Code (0)
등록된 구현이 없습니다.
Tasks
Data VisualizationDimensionality ReductionFace RecognitionImage ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring the Influence of Dimensionality Reduction on Anomaly Detection Performance in Multivariate Time Series
This paper presents an extensive empirical study on the integration of dimensionality reduction techniques with advanced unsupervised time series anomaly detection models, focusing on the MUTANT and Anomaly-Transformer m…
Anomaly DetectionDimensionality ReductionTime SeriesTime Series Analysis+1Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality
In human-level NLP tasks, such as predicting mental health, personality, or demographics, the number of observations is often smaller than the standard 768+ hidden state sizes of each layer within modern transformer-base…
Dimensionality ReductionDimensionality Reduction in Sentence Transformer Vector Databases with Fast Fourier Transform
Dimensionality reduction in vector databases is pivotal for streamlining AI data management, enabling efficient storage, faster computation, and improved model performance. This paper explores the benefits of reducing ve…
Computational EfficiencyDimensionality ReductionManagementRAG+4DimenFix: A novel meta-dimensionality reduction method for feature preservation
Dimensionality reduction has become an important research topic as demand for interpreting high-dimensional datasets has been increasing rapidly in recent years. There have been many dimensionality reduction methods with…
Dimensionality ReductionOn Geodesic Distances and Contextual Embedding Compression for Text Classification
In some memory-constrained settings like IoT devices and over-the-network data pipelines, it can be advantageous to have smaller contextual embeddings. We investigate the efficacy of projecting contextual embedding data …
ClassificationDimensionality ReductionGeneral Classificationtext-classification+1