Regularized Loss Minimizers with Local Data Perturbation: Consistency and Data Irrecoverability
We introduce a new concept, data irrecoverability, and show that the well-studied concept of data privacy is sufficient but not necessary for data irrecoverability. We show that there are several regularized loss minimization problems that can use perturbed data with theoretical guarantees of generalization, i.e., loss consistency. Our results quantitatively connect the convergence rates of the learning problems to the impossibility for any adversary for recovering the original data from perturbed observations. In addition, we show several examples where the convergence rates with perturbed data only increase the convergence rates with original data within a constant factor related to the amount of perturbation, i.e., noise.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ERM and RERM are optimal estimators for regression problems when malicious outliers corrupt the labels
We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) for regression problems with convex and $L$-Lipschitz loss functions. We consider a setting where $|\cO|$ malicious outliers conta…
regressionOn the Loss Landscape Geometry of Regularized Deep Matrix Factorization: Uniqueness and Sharpness
Weight decay is ubiquitous in training deep neural network architectures. Its empirical success is often attributed to capacity control; nonetheless, our theoretical understanding of its effect on the loss landscape and …
Piecewise Strong Convexity of Neural Networks
We study the loss surface of a feed-forward neural network with ReLU non-linearities, regularized with weight decay. We show that the regularized loss function is piecewise strongly convex on an important open set which …
image-classificationImage ClassificationLearning TheoryA Mosco sufficient condition for intrinsic stability of non-unique convex Empirical Risk Minimization
Empirical risk minimization (ERM) stability is usually studied via single-valued outputs, while convex non-strict losses yield set-valued minimizers. We identify Painlevé-Kuratowski upper semicontinuity (PK-u.s.c.) as th…
Label Noise SGD Provably Prefers Flat Global Minimizers
In overparametrized models, the noise in stochastic gradient descent (SGD) implicitly regularizes the optimization trajectory and determines which local minimum SGD converges to. Motivated by empirical studies that demon…