Learning Single Index Models in High Dimensions
Single Index Models (SIMs) are simple yet flexible semi-parametric models for classification and regression. Response variables are modeled as a nonlinear, monotonic function of a linear combination of features. Estimation in this context requires learning both the feature weights, and the nonlinear function. While methods have been described to learn SIMs in the low dimensional regime, a method that can efficiently learn SIMs in high dimensions has not been forthcoming. We propose three variants of a computationally and statistically efficient algorithm for SIM inference in high dimensions. We establish excess risk bounds for the proposed algorithms and experimentally validate the advantages that our SIM learning methods provide relative to Generalized Linear Model (GLM) and low dimensional SIM based learning methods.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k
DeepSeek-V3.2 and V4 introduce Compressed Sparse Attention (CSA): a lightning indexer (a learned scoring projection over compressed keys) scores them, the top-k are selected per query, and a sparse attention kernel reads…
On Learning High Dimensional Structured Single Index Models
Single Index Models (SIMs) are simple yet flexible semi-parametric models for machine learning, where the response variable is modeled as a monotonic function of a linear combination of features. Estimation in this conte…
Vocal Bursts Intensity PredictionHigh-dimensional Non-Gaussian Single Index Models via Thresholded Score Function Estimation
We consider estimating the parametric component of single index models in high dimensions. Compared with existing work, we do not require the covariate to be normally distributed. Utilizing Stein’s Lemma, we propose…
LEMMARobust Feature Learning for Multi-Index Models in High Dimensions
Recently, there have been numerous studies on feature learning with neural networks, specifically on learning single- and multi-index models where the target is a function of a low-dimensional projection of the input. Pr…
Hypergraph Embedding Indexing for Efficient Dense Vector Retrieval
Dense vector retrieval has become the foundation of modern semantic search, yet existing approximate nearest neighbor (ANN) indexes treat an embedding as an indivisible point in a high-dimensional space. In this work, we…