Deep Dimension Reduction for Supervised Representation Learning

2020-06-10 · Jian Huang, Yuling Jiao, Xu Liao, Jin Liu, Zhou Yu

The goal of supervised representation learning is to construct effective data representations for prediction. Among all the characteristics of an ideal nonparametric representation of high-dimensional complex data, sufficiency, low dimensionality and disentanglement are some of the most essential ones. We propose a deep dimension reduction approach to learning representations with these characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We formulate the ideal representation learning task as that of finding a nonparametric representation that minimizes an objective function characterizing conditional independence and promoting disentanglement at the population level. We then estimate the target representation at the sample level nonparametrically using deep neural networks. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero. Our extensive numerical experiments using simulated and real benchmark data demonstrate that the proposed methods have better performance than several existing dimension reduction methods and the standard deep learning models in the context of classification and regression.

📄 PDF Abstract BibTeX arXiv:2006.05865

Code (1)

Liao-Xu/DDR 공식 구현 pytorch

Tasks

Dimensionality Reduction Disentanglement Representation Learning

Unsupervised Kernel Dimension Reduction

2010-12-01 · NeurIPS 2010 12 · Meihong Wang, Fei Sha, Michael. I. Jordan

We apply the framework of kernel dimension reduction, originally designed for supervised problems, to unsupervised dimensionality reduction. In this framework, kernel-based measures of independence are used to derive low…

Classification Clustering Dimensionality Reduction General Classification

TLDR: Twin Learning for Dimensionality Reduction

2021-10-18 · Yannis Kalantidis, Carlos Lassance, Jon Almazan, Diane Larlus

Dimensionality reduction methods are unsupervised approaches which learn low-dimensional spaces where some properties of the initial space, typically the notion of "neighborhood", are preserved. Such methods usually requ…

Dimensionality Reduction Representation Learning Retrieval Self-Supervised Learning

Hybrid Quantum-Classical NLP Classification with Compact Semantic Representations: An Experimental Analysis of Representation Compression

2026-09-09 · Ali Hassan, Zijia Zhao, Maha A. Metawei arxiv

Large language and sentence-embedding models provide rich semantic representations, but their high dimensionality poses a challenge for near-term quantum machine learning (QML), where quantum circuits can process only a …

Quantum Machine Learning Dimensionality Reduction

GNUMAP: A Parameter-Free Approach to Unsupervised Dimensionality Reduction via Graph Neural Networks

2024-07-30 · Jihee You, So Won Jeong, Claire Donnat

With the proliferation of Graph Neural Network (GNN) methods stemming from contrastive learning, unsupervised node representation learning for graph data is rapidly gaining traction across various fields, from biology to…

Benchmarking Contrastive Learning Dimensionality Reduction Graph Neural Network+1

Data Dimension Reduction makes ML Algorithms efficient

2022-11-17 · Wisal Khan, Muhammad Turab, Waqas Ahmad, Syed Hasnat Ahmad 외

Data dimension reduction (DDR) is all about mapping data from high dimensions to low dimensions, various techniques of DDR are being used for image dimension reduction like Random Projections, Principal Component Analysi…

Dimensionality Reduction Representation Learning