Predicting Out-of-Distribution Error with the Projection Norm
We propose a metric -- Projection Norm -- to predict a model's performance on out-of-distribution (OOD) data without access to ground truth labels. Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels. The more the new model's parameters differ from an in-distribution model, the greater the predicted OOD error. Empirically, our approach outperforms existing methods on both image and text classification tasks and across different network architectures. Theoretically, we connect our approach to a bound on the test error for overparameterized linear models. Furthermore, we find that Projection Norm is the only approach that achieves non-trivial detection performance on adversarial examples. Our code is available at https://github.com/yaodongyu/ProjNorm.
Code (1)
Tasks
Pseudo Labeltext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Error Bounds of Projection Models in Weakly Supervised 3D Human Pose Estimation
The current state-of-the-art in monocular 3D human pose estimation is heavily influenced by weakly supervised methods. These allow 2D labels to be used to learn effective 3D human pose recovery either directly from image…
3D Human Pose EstimationMonocular 3D Human Pose EstimationPose EstimationPosition+1Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting
Fitted \(Q\)-evaluation (FQE) is a standard regression-based method for off-policy evaluation, but under distribution shift, value-function realizability alone does not ensure convergence, and existing analyses often req…
Restricted Generative Projection for One-Class Classification and Anomaly Detection
We present a simple framework for one-class classification and anomaly detection. The core idea is to learn a mapping to transform the unknown distribution of training (normal) data to a known target distribution. Crucia…
Anomaly DetectionInformativenessOne-Class ClassificationInvariance principle of random projection for the norm
Johnson-Lindenstrauss guarantees certain topological structure is preserved under random projections when project high dimensional deterministic vectors to low dimensional vectors. In this work, we try to understand how …
Point Triangulation Through Polyhedron Collapse Using the l[?] Norm
Multi-camera triangulation of feature points based on a minimisation of the overall L2 reprojection error can get stuck in suboptimal local minima or require slow global optimisation. For this reason, researchers have pr…