Bayesian inverse regression for dimension reduction with small datasets

2019-06-19 · Xin Cai, Guang Lin, Jinglai Li

We consider supervised dimension reduction problems, namely to identify a low dimensional projection of the predictors $\-x$ which can retain the statistical relationship between $\-x$ and the response variable $y$. We follow the idea of the sliced inverse regression (SIR) and the sliced average variance estimation (SAVE) type of methods, which is to use the statistical information of the conditional distribution $\pi(\-x|y)$ to identify the dimension reduction (DR) space. In particular we focus on the task of computing this conditional distribution without slicing the data. We propose a Bayesian framework to compute the conditional distribution where the likelihood function is obtained using the Gaussian process regression model. The conditional distribution $\pi(\-x|y)$ can then be computed directly via Monte Carlo sampling. We then can perform DR by considering certain moment functions (e.g. the first or the second moment) of the samples of the posterior distribution. With numerical examples, we demonstrate that the proposed method is especially effective for small data problems.

📄 PDF Abstract BibTeX arXiv:1906.08018

Code (0)

등록된 구현이 없습니다.

Tasks

Dimensionality Reduction regression Small Data Image Classification

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Federated Sufficient Dimension Reduction Through High-Dimensional Sparse Sliced Inverse Regression

2023-01-23 · Wenquan Cui, Yue Zhao, Jianjun Xu, Haoyang Cheng

Federated learning has become a popular tool in the big data era nowadays. It trains a centralized model based on data from different clients while keeping data decentralized. In this paper, we propose a federated sparse…

Dimensionality Reduction Federated Learning regression Variable Selection+1

High Dimensional Bayesian Optimization via Supervised Dimension Reduction

2019-07-21 · Miao Zhang, Huiqi Li, Steven Su

Bayesian optimization (BO) has been broadly applied to computational expensive problems, but it is still challenging to extend BO to high dimensions. Existing works are usually under strict assumption of an additive or a…

Bayesian Optimization Dimensionality Reduction Vocal Bursts Intensity Prediction

Overlapping Sliced Inverse Regression for Dimension Reduction

2018-06-23 · Ning Zhang, Zhou Yu, Qiang Wu

Sliced inverse regression (SIR) is a pioneer tool for supervised dimension reduction. It identifies the effective dimension reduction space, the subspace of significant factors with intrinsic lower dimensionality. In thi…

Dimensionality Reduction regression

Online Kernel Sliced Inverse Regression

2023-01-23 · Wenquan Cui, Yue Zhao, Jianjun Xu, Haoyang Cheng

Online dimension reduction is a common method for high-dimensional streaming data processing. Online principal component analysis, online sliced inverse regression, online kernel principal component analysis and other me…

Dimensionality Reduction regression Stochastic Optimization

Functional Inverse Regression in an Enlarged Dimension Reduction Space

2015-03-12 · Ting-Li Chen, Su-Yun Huang, Yanyuan Ma, I-Ping Tu

We consider an enlarged dimension reduction space in functional inverse regression. Our operator and functional analysis based approach facilitates a compact and rigorous formulation of the functional inverse regression …