Screening Data Points in Empirical Risk Minimization via Ellipsoidal Regions and Safe Loss Functions
We design simple screening tests to automatically discard data samples in empirical risk minimization without losing optimization guarantees. We derive loss functions that produce dual objectives with a sparse solution. We also show how to regularize convex losses to ensure such a dual sparsity-inducing property, and propose a general method to design screening tests for classification or regression based on ellipsoidal approximations of the optimal set. In addition to producing computational gains, our approach also allows us to compress a dataset into a subset of representative points.
Code (1)
Tasks
regressionSimilar Papers 제목 키워드 기반
Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization
Pairwise similarities and dissimilarities between data points might be easier to obtain than fully labeled data in real-world classification problems, e.g., in privacy-aware situations. To handle such pairwise informatio…
ClusteringGeneral ClassificationGlobal Guarantees for Enforcing Deep Generative Priors by Empirical Risk
We examine the theoretical properties of enforcing priors provided by generative deep neural networks via empirical risk minimization. In particular we consider two models, one in which the task is to invert a generative…
SGD Algorithms based on Incomplete U-statistics: Large-Scale Minimization of Empirical Risk
In many learning problems, ranging from clustering to ranking through metric learning, empirical estimates of the risk functional consist of an average over tuples (e.g., pairs or triplets) of observations, rather than o…
ClusteringMetric LearningOn Memorization and Privacy Risks of Sharpness Aware Minimization
In many recent works, there is an increased focus on designing algorithms that seek flatter optima for neural network loss optimization as there is empirical evidence that it leads to better generalization performance in…
MemorizationDistributionally Robust Safe Screening
In this study, we propose a method Distributionally Robust Safe Screening (DRSS), for identifying unnecessary samples and features within a DR covariate shift setting. This method effectively combines DR learning, a para…