Estimation of a function of low local dimensionality by deep neural networks
Deep neural networks (DNNs) achieve impressive results for complicated tasks like object detection on images and speech recognition. Motivated by this practical success, there is now a strong interest in showing good theoretical properties of DNNs. To describe for which tasks DNNs perform well and when they fail, it is a key challenge to understand their performance. The aim of this paper is to contribute to the current statistical theory of DNNs. We apply DNNs on high dimensional data and we show that the least squares regression estimates using DNNs are able to achieve dimensionality reduction in case that the regression function has locally low dimensionality. Consequently, the rate of convergence of the estimate does not depend on its input dimension $d$, but on its local dimension $d^*$ and the DNNs are able to circumvent the curse of dimensionality in case that $d^*$ is much smaller than $d$. In our simulation study we provide numerical experiments to support our theoretical result and we compare our estimate with other conventional nonparametric regression estimates. The performance of our estimates is also validated in experiments with real data.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality Reductionobject-detectionObject Detectionregressionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Intrinsic Dimensionality Estimation within Tight Localities: A Theoretical and Experimental Analysis
Accurate estimation of Intrinsic Dimensionality (ID) is of crucial importance in many data mining and machine learning tasks, including dimensionality reduction, outlier detection, similarity search and subspace clusteri…
Dimensionality ReductionOutlier DetectionProcess monitoring based on orthogonal locality preserving projection with maximum likelihood estimation
By integrating two powerful methods of density reduction and intrinsic dimensionality estimation, a new data-driven method, referred to as OLPP-MLE (orthogonal locality preserving projection-maximum likelihood estimation…
Density EstimationDimensionality ReductionFault DetectionFault DiagnosisA Theory of Nonparametric Covariance Function Estimation for Discretely Observed Data
We study nonparametric covariance function estimation for functional data observed with noise at discrete locations on a $d$-dimensional domain. Estimating the covariance function from discretely observed data is a chall…
Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis
We present a nonparametric method for outlier detection that takes full account of local variations in intrinsic dimensionality within the dataset. Using the theory of Local Intrinsic Dimensionality (LID), our 'dimension…
Outlier DetectionRandom Maxout Features
In this paper, we propose and study random maxout features, which are constructed by first projecting the input data onto sets of randomly generated vectors with Gaussian elements, and then outputing the maximum projecti…
Data VisualizationDimensionality ReductionGeneral ClassificationGeneralization Bounds