On the Spectrum of Random Features Maps of High Dimensional Data
Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper, we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gram matrix of these random feature maps, here for Gaussian mixture models of simultaneously large dimension and size. Our results are instrumental to a deeper understanding on the interplay of the nonlinearity and the statistics of the data, thereby allowing for a better tuning of random feature-based techniques.
Code (1)
Tasks
BIG-bench Machine LearningVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Rotationally Invariant Image Representation for Viewing Direction Classification in Cryo-EM
We introduce a new rotationally invariant viewing angle classification method for identifying, among a large number of Cryo-EM projection images, similar views without prior knowledge of the molecule. Our rotationally in…
ClassificationClusteringGeneral ClassificationHigh-dimensional ridge regression with random features for non-identically distributed data with a variance profile
The behavior of the random feature model in the high-dimensional regression framework has become a popular issue of interest in the machine learning literature}. This model is generally considered for feature vectors $x_…
Random matrix approach to estimation of high-dimensional factor models
In dealing with high-dimensional data sets, factor models are often useful for dimension reduction. The estimation of factor models has been actively studied in various fields. In the first part of this paper, we present…
Dimensionality ReductionTime SeriesTime Series AnalysisVocal Bursts Intensity PredictionRandom Laplace Feature Maps for Semigroup Kernels on Histograms
With the goal of accelerating the training and testing complexity of nonlinear kernel methods, several recent papers have proposed explicit embeddings of the input data into low-dimensional feature spaces, where fast lin…
Event Detectionimage-classificationImage ClassificationDemystifying the Global Convergence Puzzle of Learning Over-parameterized ReLU Nets in Very High Dimensions
This theoretical paper is devoted to developing a rigorous theory for demystifying the global convergence phenomenon in a challenging scenario: learning over-parameterized Rectified Linear Unit (ReLU) nets for very high …