Spectral Overlap and a Comparison of Parameter-Free, Dimensionality Reduction Quality Metrics
Nonlinear dimensionality reduction methods are a popular tool for data scientists and researchers to visualize complex, high dimensional data. However, while these methods continue to improve and grow in number, it is often difficult to evaluate the quality of a visualization due to a variety of factors such as lack of information about the intrinsic dimension of the data and additional tuning required for many evaluation metrics. In this paper, we seek to provide a systematic comparison of dimensionality reduction quality metrics using datasets where we know the ground truth manifold. We utilize each metric for hyperparameter optimization in popular dimensionality reduction methods used for visualization and provide quantitative metrics to objectively compare visualizations to their original manifold. In our results, we find a few methods that appear to consistently do well and propose the best performer as a benchmark for evaluating dimensionality reduction based visualizations.
Code (1)
Tasks
Dimensionality ReductionHyperparameter OptimizationSimilar Papers 제목 키워드 기반
Hyperspectral Band Selection based on Generalized 3DTV and Tensor CUR Decomposition
Hyperspectral Imaging (HSI) serves as an important technique in remote sensing. However, high dimensionality and data volume typically pose significant computational challenges. Band selection is essential for reducing s…
A novel information gain-based approach for classification and dimensionality reduction of hyperspectral images
Recently, the hyperspectral sensors have improved our ability to monitor the earth surface with high spectral resolution. However, the high dimensionality of spectral data brings challenges for the image processing. Cons…
ClassificationClassification Of Hyperspectral ImagesDimensionality ReductionLaplacian Mixture Modeling for Network Analysis and Unsupervised Learning on Graphs
Laplacian mixture models identify overlapping regions of influence in unlabeled graph and network data in a scalable and computationally efficient way, yielding useful low-dimensional representations. By combining Laplac…
ClusteringCommunity DetectionDimensionality ReductionSpectral Tempering for Embedding Compression in Dense Passage Retrieval
Dimensionality reduction is critical for deploying dense retrieval systems at scale, yet mainstream post-hoc methods face a fundamental trade-off: principal component analysis (PCA) preserves dominant variance but underu…
Dimensionality ReductionPassage RetrievalHyperspectral Images Classification and Dimensionality Reduction using spectral interaction and SVM classifier
Over the past decades, the hyperspectral remote sensing technology development has attracted growing interest among scientists in various domains. The rich and detailed spectral information provided by the hyperspectral …
ClassificationDimensionality Reduction