Supervised Manifold Learning via Random Forest Geometry-Preserving Proximities
Manifold learning approaches seek the intrinsic, low-dimensional data structure within a high-dimensional space. Mainstream manifold learning algorithms, such as Isomap, UMAP, $t$-SNE, Diffusion Map, and Laplacian Eigenmaps do not use data labels and are thus considered unsupervised. Existing supervised extensions of these methods are limited to classification problems and fall short of uncovering meaningful embeddings due to their construction using order non-preserving, class-conditional distances. In this paper, we show the weaknesses of class-conditional manifold learning quantitatively and visually and propose an alternate choice of kernel for supervised dimensionality reduction using a data-geometry-preserving variant of random forest proximities as an initialization for manifold learning methods. We show that local structure preservation using these proximities is near universal across manifold learning approaches and global structure is properly maintained using diffusion-based algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionSupervised dimensionality reductionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Random Forest-Supervised Manifold Alignment
Manifold alignment is a type of data fusion technique that creates a shared low-dimensional representation of data collected from multiple domains, enabling cross-domain learning and improved performance in downstream ta…
Geometry- and Accuracy-Preserving Random Forest Proximities
Random forests are considered one of the best out-of-the-box classification and regression algorithms due to their high level of predictive performance with relatively little tuning. Pairwise proximities can be computed …
Data VisualizationImputationOutlier DetectionregressionRandom Forest Autoencoders for Guided Representation Learning
Extensive research has produced robust methods for unsupervised data visualization. Yet supervised visualization$\unicode{x2013}$where expert labels guide representations$\unicode{x2013}$remains underexplored, as most su…
Data VisualizationDimensionality ReductionRepresentation LearningForest-Guided Semantic Transport for Label-Supervised Manifold Alignment
Label-supervised manifold alignment bridges the gap between unsupervised and correspondence-based paradigms by leveraging shared label information to align multimodal datasets. Still, most existing methods rely on Euclid…
An RKHS Perspective on Tree Ensembles
Random Forests and Gradient Boosting are among the most effective algorithms for supervised learning on tabular data. Both belong to the class of tree-based ensemble methods, where predictions are obtained by aggregating…
Ensemble Learning