Predicting Generalization in Deep Learning via Local Measures of Distortion
We study generalization in deep learning by appealing to complexity measures originally developed in approximation and information theory. While these concepts are challenged by the high-dimensional and data-defined nature of deep learning, we show that simple vector quantization approaches such as PCA, GMMs, and SVMs capture their spirit when applied layer-wise to deep extracted features giving rise to relatively inexpensive complexity measures that correlate well with generalization performance. We discuss our results in 2020 NeurIPS PGDL challenge.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Investigations on the Influence of Combined Inter-Aural Cue Distortions on Overall Audio Quality
There is a considerable interest in developing algorithms that can predict audio quality of perceptually coded signals to avoid the cost of extensive listening tests during development time. While many established algori…
Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy
The mainstream approach to image quality assessment has centered around accurately modeling the single most relevant strategy employed by the human visual system (HVS) when judging image quality (e.g., detecting visible…
Full reference image quality assessmentFull-Reference Image Quality AssessmentImage Quality AssessmentFull-reference image quality assessment by combining global and local distortion measures
Full-reference image quality assessment (FR-IQA) techniques compare a reference and a distorted/test image and predict the perceptual quality of the test image in terms of a scalar value representing an objective score. …
Full reference image quality assessmentFull-Reference Image Quality AssessmentImage Quality AssessmentLocal DistortionGeneralization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark
Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benchmark of Jiang et al. (2020) evaluated many generalization measures, but …
Predicting Deep Neural Network Generalization with Perturbation Response Curves
The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of prediction tasks. However, despite these successes, the recent Predicting Generalization in Deep Learning (PGDL) NeurIP…