Penalized versus constrained generalized eigenvalue problems

2014-10-22 · Irina Gaynanova, James Booth, Martin T. Wells

We investigate the difference between using an $\ell_1$ penalty versus an $\ell_1$ constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an $\ell_1$ penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an $\ell_1$ constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of an $\ell_1$ constraint in the context of discriminant analysis and principal component analysis.

📄 PDF Abstract BibTeX arXiv:1410.6131

Code (0)

등록된 구현이 없습니다.

Tasks

Variable Selection

Eigenvalue and Generalized Eigenvalue Problems: Tutorial

2019-03-25 · Benyamin Ghojogh, Fakhri Karray, Mark Crowley

This paper is a tutorial for eigenvalue and generalized eigenvalue problems. We first introduce eigenvalue problem, eigen-decomposition (spectral decomposition), and generalized eigenvalue problem. Then, we mention the o…

BIG-bench Machine Learning Matrix Factorization / Decomposition

Sparse Generalized Eigenvalue Problem via Smooth Optimization

2014-08-28 · Junxiao Song, Prabhu Babu, Daniel P. Palomar

In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves m…

p-PSO: A Penalized Particle Swarm Optimization Technique for Finding D-Optimal Designs with Mixed Factors in Generalized Linear Models

2026-06-14 · Shrabanti Chowdhury, Abhyuday Mandal arxiv

Finding D-optimal designs for generalized linear models (GLMs) is challenging due to the dependence of the Fisher information matrix on unknown parameters and the lack of closed-form solutions, particularly when input fa…

Alternating direction method of multipliers for penalized zero-variance discriminant analysis

2014-01-21 · Brendan P. W. Ames, Mingyi Hong

We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuris…

feature selection General Classification Time Series Time Series Analysis+1

Grouped Variable Selection for Generalized Eigenvalue Problems

2021-05-28 · Jonathan Dan, Simon Geirnaert, Alexander Bertrand

Many problems require the selection of a subset of variables from a full set of optimization variables. The computational complexity of an exhaustive search over all possible subsets of variables is, however, prohibitive…