Dimension Reduction with Non-degrading Generalization
Visualizing high dimensional data by projecting them into two or three dimensional space is one of the most effective ways to intuitively understand the data's underlying characteristics, for example their class neighborhood structure. While data visualization in low dimensional space can be efficient for revealing the data's underlying characteristics, classifying a new sample in the reduced-dimensional space is not always beneficial because of the loss of information in expressing the data. It is possible to classify the data in the high dimensional space, while visualizing them in the low dimensional space, but in this case, the visualization is often meaningless because it fails to illustrate the underlying characteristics that are crucial for the classification process. In this paper, the performance-preserving property of the previously proposed Restricted Radial Basis Function Network in reducing the dimension of labeled data is explained. Here, it is argued through empirical experiments that the internal representation of the Restricted Radial Basis Function Network, which during the supervised learning process organizes a visualizable two dimensional map, does not only preserve the topographical structure of high dimensional data but also captures their class neighborhood structures that are important for classifying them. Hence, unlike many of the existing dimension reduction methods, the Restricted Radial Basis Function Network offers two dimensional visualization that is strongly correlated with the classification process.
Code (0)
등록된 구현이 없습니다.
Tasks
Data VisualizationDimensionality ReductionGeneral ClassificationSimilar Papers 제목 키워드 기반
Kernelized Diffusion maps
Spectral clustering and diffusion maps are celebrated dimensionality reduction algorithms built on eigen-elements related to the diffusive structure of the data. The core of these procedures is the approximation of a Lap…
ClusteringDimensionality ReductionAdaptive Metric Dimensionality Reduction
We study adaptive data-dependent dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spac…
Dimensionality ReductionGeneralization BoundsLearning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions
Generalization across different environments with the same tasks is critical for successful applications of visual reinforcement learning (RL) in real scenarios. However, visual distractions -- which are common in real s…
Reinforcement Learning (RL)Foundations of Coupled Nonlinear Dimensionality Reduction
In this paper we introduce and analyze the learning scenario of \emph{coupled nonlinear dimensionality reduction}, which combines two major steps of machine learning pipeline: projection onto a manifold and subsequent su…
Dimensionality ReductionGeneralization BoundsSupervised dimensionality reductionEffective Sample Size, Dimensionality, and Generalization in Covariate Shift Adaptation
In supervised learning, training and test datasets are often sampled from distinct distributions. Domain adaptation techniques are thus required. Covariate shift adaptation yields good generalization performance when dom…
Dimensionality ReductionDomain Adaptationfeature selectionGeneralization Bounds