Simultaneous Best Subset Selection and Dimension Reduction via Primal-Dual Iterations
Sparse reduced rank regression is an essential statistical learning method. In the contemporary literature, estimation is typically formulated as a nonconvex optimization that often yields to a local optimum in numerical computation. Yet, their theoretical analysis is always centered on the global optimum, resulting in a discrepancy between the statistical guarantee and the numerical computation. In this research, we offer a new algorithm to address the problem and establish an almost optimal rate for the algorithmic solution. We also demonstrate that the algorithm achieves the estimation with a polynomial number of iterations. In addition, we present a generalized information criterion to simultaneously ensure the consistency of support set recovery and rank estimation. Under the proposed criterion, we show that our algorithm can achieve the oracle reduced rank estimation with a significant probability. The numerical studies and an application in the ovarian cancer genetic data demonstrate the effectiveness and scalability of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionregressionSimilar Papers 제목 키워드 기반
A Consistent and Scalable Algorithm for Best Subset Selection in Single Index Models
Analysis of high-dimensional data has led to increased interest in both single index models (SIMs) and best subset selection. SIMs provide an interpretable and flexible modeling framework for high-dimensional data, while…
Model SelectionregressionOn Subspace Approximation and Subset Selection in Fewer Passes by MCMC Sampling
We consider the problem of subset selection for $\ell_{p}$ subspace approximation, i.e., given $n$ points in $d$ dimensions, we need to pick a small, representative subset of the given points such that its span gives $(1…
Feature Selection Using Reinforcement Learning
With the decreasing cost of data collection, the space of variables or features that can be used to characterize a particular predictor of interest continues to grow exponentially. Therefore, identifying the most charact…
Dimensionality Reductionfeature selectionreinforcement-learningReinforcement Learning+1Solving the Best Subset Selection Problem via Suboptimal Algorithms
Best subset selection in linear regression is well known to be nonconvex and computationally challenging to solve, as the number of possible subsets grows rapidly with increasing dimensionality of the problem. As a resul…
CPUA determinantal point process for column subset selection
Dimensionality reduction is a first step of many machine learning pipelines. Two popular approaches are principal component analysis, which projects onto a small number of well chosen but non-interpretable directions, an…
Dimensionality Reductionfeature selection